Intelligent light compensation dynamic control method and system based on scene recognition

Abstract

The application discloses a scene recognition-based intelligent light supplementing dynamic control method and system, relates to the technical field of intelligent light supplementing control, and comprises the following steps: setting a turning light supplementing device, the initial position of the turning light supplementing device needs to be the same as the direction of a cmos image collector; constructing a scene classification, and adding multiple classification results to a scene data set; collecting a current scene image by using the cmos image collector to obtain scene image data, and judging the scene image data according to the scene data set; obtaining a light supplementing area according to a scene judgment result, and obtaining light supplementing area position information of the turning light supplementing device according to the light supplementing area; and turning the turning light supplementing device to compensate according to the light supplementing area position information, so that the problem that operation is complicated and it is difficult to operate due to the fact that light supplementing control is not intelligent and complete enough in the prior art is solved, and efficient and accurate management and control of the final light supplementing work are realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent supplemental lighting control technology, specifically to an intelligent supplemental lighting dynamic control method and system based on scene recognition. Background Technology

[0002] With the rapid development of society and economy and the continuous improvement of people's demand for photography, intelligent lighting for photography has gradually developed. Flash units are frequently used lighting devices in photography and play an important role. Existing lighting devices for photography are not only bulky, but also require manual settings throughout the process, making them difficult for non-professionals to use. Furthermore, the scenarios requiring lighting are often complex, and learning them also has a certain barrier to entry. Therefore, the emergence of intelligent lighting control is of great significance.

[0003] Existing technologies for supplemental lighting control suffer from problems such as cumbersome operation and difficulty in learning due to insufficient intelligence and completeness, ultimately making it impossible to control supplemental lighting work efficiently and accurately. Summary of the Invention

[0004] This application provides a scene recognition-based intelligent supplementary lighting dynamic control method and system to solve the problems of cumbersome operation and difficulty in learning existing supplementary lighting control due to insufficient intelligence and completeness, which ultimately makes it impossible to efficiently and accurately control supplementary lighting work.

[0005] In view of the above problems, this application provides a method for intelligent supplementary lighting dynamic control based on scene recognition.

[0006] Firstly, this application provides a scene recognition-based intelligent supplementary lighting dynamic control method, the method comprising: setting up a directional supplementary lighting device, the initial position of which needs to be the same as the direction of a CMOS image acquisition device, wherein the directional supplementary lighting device and the CMOS image acquisition device are at the same horizontal position in space, and the directional supplementary lighting device and the CMOS image acquisition device are fixed on the same base, and the viewing angle error between the CMOS image acquisition device and the directional supplementary lighting device is within a preset range; constructing a scene classification and adding multiple classification results to a scene dataset; acquiring a scene image using the CMOS image acquisition device to obtain scene image data, and judging the scene image data according to the scene dataset; obtaining a supplementary lighting area based on the scene judgment result, and obtaining the supplementary lighting area position information of the directional supplementary lighting device based on the supplementary lighting area; and rotating the directional supplementary lighting device to compensate according to the supplementary lighting area position information.

[0007] Secondly, this application provides an intelligent supplementary lighting dynamic control system based on scene recognition. The system includes: a supplementary lighting device module: setting a directional supplementary lighting device, the initial position of which must be the same as the direction of the CMOS image acquisition device, wherein the directional supplementary lighting device and the CMOS image acquisition device are at the same horizontal position in space, and the directional supplementary lighting device and the CMOS image acquisition device are fixed on the same base, and the viewing angle error between the CMOS image acquisition device and the directional supplementary lighting device is within a preset range; a scene classification module: constructing scene classification and adding multiple classification results to a scene dataset; an image acquisition module: using the CMOS image acquisition device to acquire the current scene image to obtain scene image data, and judging the scene image data according to the scene dataset; a supplementary lighting area module: obtaining the supplementary lighting area according to the scene judgment result, and obtaining the supplementary lighting area position information of the directional supplementary lighting device according to the supplementary lighting area; and a supplementary lighting position module: rotating the directional supplementary lighting device to compensate according to the supplementary lighting area position information.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] The intelligent supplementary lighting dynamic control method and system based on scene recognition provided in this application embodiment, by setting a directional supplementary lighting device, the initial position of which must be the same as the lens direction, then constructing scene classification, adding multiple classification results to the scene dataset, then acquiring the current scene image to obtain scene image data, judging the scene image data according to the scene dataset, obtaining the supplementary lighting area according to the scene judgment result, obtaining the supplementary lighting area position information of the directional supplementary lighting device according to the supplementary lighting area, and finally rotating the directional supplementary lighting device to compensate according to the supplementary lighting area position information, solves the problem of cumbersome operation and difficulty in learning existing supplementary lighting control due to insufficient intelligence and incompleteness. It enables efficient and accurate control of supplementary lighting work, improves the accuracy of supplementary lighting position, and lowers the usage threshold of supplementary lighting operation. Attached Figure Description

[0010] Figure 1 This application provides a schematic flowchart of an intelligent supplementary lighting dynamic control method based on scene recognition;

[0011] Figure 2 This application provides a schematic diagram of the intelligent supplementary lighting dynamic control system structure based on scene recognition.

[0012] Explanation of reference numerals in the attached diagram: 11. Fill light device module; 12. Scene classification module; 13. Image acquisition module; 14. Fill light area module; 15. Fill light position module. Detailed Implementation

[0013] In portrait photography, relying solely on natural light or a single scene light source can result in various image effects and shadows, necessitating scene lighting. However, for most non-professionals, lighting correction is a complex and cumbersome process. This application provides an intelligent dynamic lighting control method and system based on scene recognition. It involves setting up a directional lighting device, whose initial position must be aligned with the lens direction. Scene classification is then constructed, and multiple classification results are added to a scene dataset. Next, the current scene image is acquired, obtaining scene image data. The scene image data is then judged based on the scene dataset, and the lighting area is determined according to the scene judgment result. The position information of the directional lighting device's lighting area is obtained based on the lighting area, and finally, the directional lighting device is rotated to compensate according to the lighting area position information. This solves the problems of cumbersome operation and difficulty in learning existing lighting control methods due to insufficient intelligence and completeness, enabling efficient and precise control of lighting.

[0014] Example 1

[0015] like Figure 1 As shown, this application provides a method and system for intelligent supplemental lighting dynamic control based on scene recognition. The method includes:

[0016] A directional supplementary lighting device is set up. The initial position of the directional supplementary lighting device should be the same as the direction of the CMOS image acquisition device. The directional supplementary lighting device and the CMOS image acquisition device are at the same horizontal position in space, and the directional supplementary lighting device and the CMOS image acquisition device are fixed on the same base. The viewing angle error between the CMOS image acquisition device and the directional supplementary lighting device is within a preset range.

[0017] A directional fill light is a portable, battery-powered flash unit that can be freely rotated and is mounted on the camera's hot shoe. Its rotation range matches the camera's field of view, meaning it can illuminate any point in the frame. When installing the directional fill light, it must be returned to its initial position, positioned horizontally with the CMOS image sensor, maintaining a relative distance between them. This relative position must be maintained even when either the directional fill light or the CMOS image sensor moves, ensuring stability of the image captured by the CMOS sensor. If the relative position deviates due to movement, the directional fill light may fail to rotate to the designated position for fill light application. The setup of the directional fill light provides a foundation for subsequent scene-specific fill light adjustments.

[0018] Build scene classification and add multiple classification results to the scene dataset;

[0019] Since there are many scenarios requiring supplemental lighting, and different scenarios require different supplemental lighting methods, analyzing and supplementing each image individually is too computationally wasteful. Furthermore, the supplemental lighting methods for the same scene may differ, causing significant variations and resulting in inconsistent quality of the final supplemental-lit images. Therefore, it is necessary to uniformly classify the scenarios requiring supplemental lighting. Based on the classification results, the scene features are extracted, and the supplemental lighting characteristics of the scene are analyzed. Supplemental lighting is then applied to the current scene based on these characteristics, effectively saving computational resources and ensuring effective control over the overall quality of the supplemental-lit images. First, the scene is classified according to the direction of illumination from the light source to the subject. The shadow effects created by the light source on the subject are analyzed to obtain overall shadow characteristics, and the scene category is determined based on these shadow characteristics. Next, the overall brightness and contrast of the image are analyzed. Contrast is categorized as either too strong or too weak. Too strong contrast means that dark areas are too bright and bright areas are too dark, while too weak contrast means that the entire image is relatively dark, with no discernible difference between bright and dark areas. In this case, overall supplemental lighting is required. The above situation is analyzed to extract scene image features, construct scene classification, and further construct scene datasets from the classification results. The construction of scene datasets can effectively save computing power and effectively control the overall quality of supplementary lighting image results.

[0020] The current scene image is acquired using the CMOS image acquisition device to obtain scene image data, and the scene image data is judged based on the scene dataset;

[0021] The process involves acquiring an image of the desired scene using a camera, treating this scene as scene image data, and then classifying and analyzing this data. Image features are extracted and analyzed, and feature matching is used to categorize the scene image data. The matched scene categories are then labeled according to their features, resulting in a labeling result. The scene classification feature labels contain information about the supplementary lighting area. By analyzing the scene image data based on these feature labels, the corresponding supplementary lighting information can be obtained. The scene image data judgment results provide the data foundation for subsequently identifying the supplementary lighting area.

[0022] The supplementary lighting area is obtained based on the scene judgment result, and the position information of the supplementary lighting area of ​​the turning supplementary lighting device is obtained based on the supplementary lighting area.

[0023] Scene classification feature identifiers contain supplementary lighting information. Feature analysis is performed on scene image data based on these identifiers to extract image features. These features are then matched with the image features, and supplementary lighting features are obtained based on the image feature identifiers. Supplementary lighting analysis is performed on the scene image data using these features, and supplementary lighting regions are identified according to these features, resulting in a supplementary lighting dataset. This dataset includes supplementary lighting region location information and supplementary lighting feature data. Supplementary lighting feature data refers to the supplementary lighting image features obtained after supplementing lighting in the current scene classification. These features are compared with the scene image data to obtain difference values, which are then output as supplementary lighting feature data. After filtering the supplementary lighting feature data, the effect of supplementary lighting on the scene image data can be clearly analyzed. Supplementary lighting adjustments can be made based on the calculated effect, and the adjustment data is fed back to the supplementary lighting region location information for further adjustment. Supplementary lighting region location information refers to the location where supplementary lighting is needed, determined by analyzing the scene image based on the supplementary lighting feature data. This allows for the control of the supplementary lighting device to apply supplementary lighting to that location.

[0024] Acquiring supplementary lighting feature data can greatly reduce the time required for supplementary lighting adjustments. The original process of first supplementing the image with supplementary lighting, then analyzing the supplementary lighting adjustments, and finally adjusting the supplementary lighting based on the analysis is optimized into a single data processing step. Based on the calculation, the scene after supplementary lighting can be roughly known, and optimization can be performed based on the scene obtained from the calculation, saving a lot of time and thus improving overall efficiency.

[0025] The rotating supplementary lighting device is rotated to compensate according to the position information of the supplementary lighting area.

[0026] The location information of the supplementary lighting area is sent to the steering control center. The steering control center locates the supplementary lighting area based on this information, determines its physical spatial position, and then determines the steering angle of the directional supplementary lighting device. The device then turns accordingly. Once the device has turned, it waits for the shutter command. When the shutter is triggered, the directional supplementary lighting device fires a flash, with the flash parameters adjusted synchronously according to the camera's shutter speed and other settings.

[0027] Furthermore, this application also includes:

[0028] The scene dataset includes front backlighting scenes, side backlighting scenes, opposite lighting scenes, and low-light scenes;

[0029] Scene features are extracted from the positive backlighting scene, the side backlighting scene, the opposite light scene, and the lack of light scene to obtain positive backlighting scene features, side backlighting scene features, opposite light scene features, and lack of light scene features;

[0030] Add the features of the positive backlighting scene, the side backlighting scene, the opposite light scene, and the lack of light scene to the scene dataset.

[0031] Scene characteristics are categorized into direct backlighting, side backlighting, contrasting light, and low-lighting. Direct backlighting refers to a scene where the subject is directly in front of the light source, creating a large contrast between their face and the background. Because the human eye has a high dynamic range, it can generally discern details in both bright and dark areas when faced with objects with strong contrast. However, the dynamic range of imaging devices like cameras is narrower than the human eye. If the dark areas are exposed, highlights will be blown out, resulting in a loss of detail in those areas, appearing completely white. Conversely, if the bright areas are exposed, the dark areas will lose detail. Therefore, in direct backlighting scenes, it's necessary to add fill light to the subject's face. Adding fill light is the best solution to reduce the contrast between the two. If the light is coming from the side of the subject, shadows will appear on their face, creating a yin-yang effect. This scene is called side backlighting, and it's necessary to add fill light to the side that is in shadow to brighten the subject's face. The lighting becomes softer. In outdoor shooting, situations where the background is brighter than the subject are darker are called contrasting lighting scenes. For example, in an outdoor scene on a cloudy day, to show the layers of clouds, it is necessary to artificially increase the light ratio between the subject and the environment. By using fill light to increase the brightness of the subject and properly exposing the subject, the sky will darken, showing rich layers of detail, while the subject will not lose detail due to overexposure. Under-lit scenes refer to situations where the overall scene is relatively dark, and the details of the background and the subject cannot be obtained. For example, in an abandoned house, light can only enter the room through a narrow window, and the overall environment becomes dim due to insufficient light, making it impossible to see the background and the subject. In this case, fill light is needed to not only show the details of the subject and the background, but also avoid the destruction of the dark and gloomy image atmosphere caused by overexposure, thus deepening the theme. Scene classification is used to extract image features, obtaining unified features for each scene. These scene features allow us to determine whether the lighting is applied to a person or the scene itself, and if lighting is applied to a person, the specific location of the lighting can be determined using these features. The acquired scene features are added to a scene dataset, establishing a mapping relationship with the scene classification. There is a one-to-one correspondence between scene classifications and scene features. Obtaining this scene dataset provides the data foundation for subsequent scene image analysis and the identification of lighting areas.

[0032] Furthermore, this application also includes:

[0033] Distortion is determined on the scene image data to obtain the distortion type result, wherein the distortion type is divided into radial distortion and tangential distortion;

[0034] Based on the distortion type result, the scene image data is distorted to obtain a repaired scene image;

[0035] The location of the fill light area in the image is obtained from the restored scene image;

[0036] The image fill light area position is converted into the fill light area position information of the steering fill light device and then output.

[0037] During image acquisition, deviations in lens shape, manufacturing precision, and assembly processes can cause image distortion, leading to stretching or compression that distorts the original image. This results in discrepancies between the required and actual illumination areas when determining the supplementary lighting position for the distorted image. Distortion correction is necessary, followed by localization analysis of the corrected image to minimize errors in determining the supplementary lighting position. Lens distortion is categorized into radial and tangential distortion. Radial distortion, caused by the lens shape, is distributed along the lens radius. The distortion is weaker closer to the center and stronger further away. Radial distortion is further divided into barrel and pincushion distortion. Tangential distortion arises from the lens itself not being parallel to the camera sensor plane or image plane, caused by installation deviations between the lens and the lens module. Radial and tangential distortion together constitute the distortion types. Distortion repair is performed on the scene image based on the distortion type results. The scene image data is represented using a two-dimensional coordinate system and converted into pixel coordinates. The distortion restoration formula is used to calculate the pixel coordinate data, and the calculated coordinate data results are then transformed to obtain the repaired scene image. The repaired scene image is divided into illuminated and non-illuminated areas. The directional lighting device needs to acquire the position of the illuminated area, and the position of the illuminated area in the image is converted into position information in actual space. The directional lighting device then provides illumination according to the position information in actual space. This method of acquiring the repaired scene image can minimize the positioning error of the directional lighting device caused by lens distortion, making the illumination more accurate and further improving the illumination effect.

[0038] Furthermore, this application also includes:

[0039] x0=xP(1+k1r 2 +k2r 4 +k3r 6 )

[0040] y0=yP(1+k1r 2 +k2r 4 +k3r 6 )

[0041] The scene image data of the radial distortion type is repaired, where (x0, y0) is the original position of the distorted point on the scene image data, (x, y) is the new position after distortion repair, r is the radius with the optical axis center as the origin, k1, k2, k3 are the first three terms of the Taylor series expansion, P is the distortion coefficient, and the corresponding distortion coefficient is selected according to different lenses. The optical axis center is the point in the image center that has not been distorted.

[0042] The scene image data with tangential distortion is subjected to trapezoidal correction, and then processed according to the radial distortion type.

[0043] For radial distortion scene image data restoration, the coordinate points of the scene image data are input into the above formula to obtain the corrected coordinate points. After organizing the corrected coordinate points of the entire image, the restored scene image is obtained. For tangential scene image data restoration, the tangential distortion scene image data is first subjected to trapezoidal correction, and then binarized. The binarized image data is then used to detect rectangles in the image data using a rectangle detection algorithm. The coordinates of the four points of the rectangles in the image are stretched and corrected to obtain the trapezoidally corrected tangential distortion scene image data. This image data is then processed using the radial distortion type to obtain the restored scene image. In the distortion restoration formula, for lenses with large distortion, such as fisheye lenses, a third term K3 can be added for restoration. For general lenses, K3 is not necessary. In the coordinate graph after restoration using the formula, the farther away from the optical center, the greater the radial displacement, i.e., the greater the distortion. Near the optical center, there is almost no translation. After distortion correction of the scene image data, the position of the supplementary light extracted from the scene image data can be more accurate, which reduces the deviation of the subsequent turning flash device from the required flash and further improves the final imaging effect.

[0044] Furthermore, this application also includes:

[0045] The scene image data is divided according to the scene characteristics to obtain problem image regions and non-problem image regions;

[0046] Feature extraction is performed on the problematic image region and the non-problematic image region respectively to obtain the features of the problematic image region and the features of the non-problematic image region;

[0047] The problem image region features and the non-problem image region features are matched with the scene features to obtain the matching results;

[0048] The supplementary lighting area is obtained based on the matching result.

[0049] The restoration scene image is divided into image regions based on scene features. These features include those for areas requiring additional lighting and those for areas not requiring additional lighting. The image regions selected based on the features of the areas requiring additional lighting are designated as problem image regions, while those selected based on the features of the areas not requiring additional lighting are designated as non-problem image regions. Problem image region features and non-problem image region features should not overlap. If overlap is found, the scene image data is reclassified. The obtained problem image region features and non-problem image region features are matched with the scene features to obtain the location information of the additional lighting regions within the scene features; these are the additional lighting regions. Obtaining the additional lighting regions provides the data foundation for subsequent positioning of the additional lighting device.

[0050] Furthermore, this application also includes:

[0051] Set a matching threshold;

[0052] The matching results are determined based on a matching degree threshold. If the matching results meet the matching degree threshold, the supplementary lighting area is obtained. If not, the features of the problem image area and the non-problem image area of ​​the matching results are re-extracted to obtain updated feature extraction results.

[0053] The updated feature extraction results are then subjected to feature matching, and the updated supplementary lighting area is obtained based on the feature matching results.

[0054] To optimize scene matching results, when acquiring the supplementary lighting area from the repaired scene image, errors in scene classification may lead to acquisition failures or even failures. Verification is needed between the problematic and non-problematic image regions and the scene classification authentication results. If the matching value is low, it indicates that the repaired scene image does not conform to the current scene classification, and the supplementary lighting area cannot be acquired according to the features of the current scene classification; the acquired result is unusable. A matching degree threshold needs to be set. When the matching degree reaches a certain value, it indicates that the current scene classification is qualified. When the matching degree is lower than the set threshold, the repaired scene image needs to be reclassified, and the features of the reclassified problematic and non-problematic image regions need to be re-extracted to obtain updated feature extraction results. Feature matching is then performed, and the matching degree is verified again. This process is repeated until the matching degree reaches the threshold. Finally, based on the feature matching results, the supplementary lighting location area in the scene features is obtained, and the updated supplementary lighting area is obtained based on the supplementary lighting location area. Obtaining the updated supplementary lighting area can further improve the accuracy of the supplementary lighting device and ensure the final imaging effect.

[0055] Example 2

[0056] Based on the same inventive concept as the scene recognition-based intelligent supplementary lighting dynamic control method in the foregoing embodiments, such as Figure 2 As shown, this application provides an intelligent supplementary lighting dynamic control system based on scene recognition, the system comprising:

[0057] Fill light device module 11: The fill light device module 11 is used to set up a directional fill light device. The initial position of the directional fill light device must be the same as the direction of the CMOS image acquisition device. The directional fill light device and the CMOS image acquisition device are at the same horizontal position in space, and the directional fill light device and the CMOS image acquisition device are fixed on the same base. The viewing angle error between the CMOS image acquisition device and the directional fill light device is within a preset range.

[0058] Scene classification module 12: The scene classification module 12 is used to construct scene classification and add multiple classification results to the scene dataset;

[0059] Image acquisition module 13: The image acquisition module 13 is used to acquire the current scene image using the CMOS image acquisition device to obtain scene image data, and to judge the scene image data based on the scene dataset;

[0060] Fill light area module 14: The fill light area module 14 is used to obtain the fill light area according to the scene judgment result, and to obtain the fill light area position information of the turning fill light device according to the fill light area;

[0061] Complementary lighting position module 15: The supplementary lighting position module 15 is used to rotate the steering supplementary lighting device to perform compensation according to the position information of the supplementary lighting area.

[0062] Furthermore, the system also includes:

[0063] Scene Dataset Module: The scene dataset module is used to collect scene datasets including front backlighting scenes, side backlighting scenes, opposite lighting scenes, and low-light scenes;

[0064] Scene feature module: The scene feature module is used to extract scene features from the front backlight scene, the side backlight scene, the opposite light scene and the lack of light scene, and obtain front backlight scene features, side backlight scene features, opposite light scene features and lack of light scene features;

[0065] Scene dataset addition module: The scene dataset addition module is used to add the front backlight scene features, the side backlight scene features, the opposite light scene features and the lack of light scene features to the scene dataset.

[0066] Furthermore, the system also includes:

[0067] Distortion Detection Module: The distortion detection module is used to detect distortion in the scene image data and obtain distortion type results, wherein the distortion type is divided into radial distortion and tangential distortion;

[0068] Distortion Repair Module: The distortion repair module is used to repair the scene image data according to the distortion type result to obtain a repaired scene image;

[0069] Fill light area location module: The fill light area location module is used to obtain the location of the fill light area in the image based on the repair scene image;

[0070] Fill light area location information module: The fill light area location information module is used to convert the image fill light area location into the fill light area location information of the turning fill light device, and output it.

[0071] Furthermore, the system also includes:

[0072] x0=x(1+k1r 2 +k2r 4 +k3r 6 )

[0073] y0=y(1+k1r 2 +k2r 4 +k3r 6 )

[0074] Scene image data distortion repair module: The scene image data distortion repair module is used to repair the scene image data of the radial distortion type, where (x0, y0) is the original position of the distorted point on the scene image data, (x, y) is the new position after distortion repair, r is the radius with the point at the center of the optical axis as the origin, k1, k2, k3 are the first three terms of the Taylor series expansion, and the center of the optical axis is the point at the center of the image that has not been distorted;

[0075] Trapezoidal correction module: The trapezoidal correction module is used to perform trapezoidal correction on the scene image data with tangential distortion, and then process it according to the radial distortion type.

[0076] Furthermore, the system also includes:

[0077] Scene image data segmentation module: The scene image data segmentation module is used to segment the scene image data according to the scene features to obtain problem image regions and non-problem image regions;

[0078] Problem and non-problem image feature extraction module: The problem and non-problem image feature extraction module is used to extract features from the problem image region and the non-problem image region respectively, and obtain the problem image region features and the non-problem image region features;

[0079] Problem and non-problem image feature matching module: The problem and non-problem image feature matching module is used to match the features of the problem image region and the features of the non-problem image region with the scene features to obtain the matching result;

[0080] Fill light area acquisition module: The fill light area acquisition module is used to acquire the fill light area according to the matching result.

[0081] Furthermore, the system also includes:

[0082] Matching degree threshold setting module: The matching degree threshold setting module is used to set the matching degree threshold;

[0083] Matching degree threshold result determination module: The matching degree threshold result determination module is used to determine the matching result according to the matching degree threshold. If the matching result meets the matching degree threshold, the supplementary lighting area is obtained; if not, the features of the problem image area and the non-problem image area of ​​the matching result are re-extracted to obtain updated feature extraction results.

[0084] Updated fill light region module: The updated fill light region module is used to perform feature matching on the updated feature extraction results and obtain the updated fill light region based on the feature matching results.

[0085] Through the foregoing detailed description of the scene recognition-based intelligent supplementary lighting dynamic control method, those skilled in the art can clearly understand the scene recognition-based intelligent supplementary lighting dynamic control method in this embodiment. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section description.

[0086] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent supplemental lighting dynamic control based on scene recognition, characterized in that, The method includes: A directional supplementary lighting device is set up. The initial position of the directional supplementary lighting device should be the same as the direction of the CMOS image acquisition device. The directional supplementary lighting device and the CMOS image acquisition device are at the same horizontal position in space, and the directional supplementary lighting device and the CMOS image acquisition device are fixed on the same base. The viewing angle error between the CMOS image acquisition device and the directional supplementary lighting device is within a preset range. Build scene classification and add multiple classification results to the scene dataset; The current scene image is acquired using the CMOS image acquisition device to obtain scene image data, and the scene image data is classified according to the scene dataset; The supplementary lighting area is obtained based on the scene classification result, and the position information of the supplementary lighting area of ​​the turning supplementary lighting device is obtained based on the supplementary lighting area. The steering supplementary lighting device is rotated to compensate according to the position information of the supplementary lighting area; The scene dataset includes front backlighting scenes, side backlighting scenes, opposite light scenes, and low light scenes; Scene features are extracted from the positive backlighting scene, the side backlighting scene, the opposite light scene, and the lack of light scene to obtain positive backlighting scene features, side backlighting scene features, opposite light scene features, and lack of light scene features; Add the features of the positive backlighting scene, the side backlighting scene, the opposite light scene, and the lack of light scene to the scene dataset; Based on the scene classification results, the fill light area is obtained, specifically including: The scene image data is classified according to the scene characteristics to obtain problem image regions and non-problem image regions; Feature extraction is performed on the problematic image region and the non-problematic image region respectively to obtain the features of the problematic image region and the features of the non-problematic image region; The problem image region features and the non-problem image region features are matched with the scene features to obtain the matching results; The supplementary lighting area is obtained based on the matching result; Specifically, the matching of problem image region features and non-problem image region features with the scene features to obtain matching results includes: Set a matching threshold; The matching results are determined based on the matching degree threshold. Matching results that meet the matching degree threshold are used to obtain the supplementary lighting area; otherwise, the scene image data is reclassified, and the features of the problem image area and the non-problem image area after classification are re-extracted to obtain updated feature extraction results. The updated feature extraction results are then subjected to feature matching, and the updated supplementary lighting area is obtained based on the feature matching results.

2. The method as described in claim 1, characterized in that, The method for obtaining the location information of the supplementary lighting area based on the supplementary lighting area includes: Distortion is determined on the scene image data to obtain the distortion type result, wherein the distortion type is divided into radial distortion and tangential distortion; Based on the distortion type result, the scene image data is distorted to obtain a repaired scene image; The location of the fill light area in the image is obtained from the restored scene image; The image fill light area position is converted into the fill light area position information of the steering fill light device and then output.

3. The method as described in claim 2, characterized in that, The method further includes performing distortion repair on the scene image data based on the distortion type result to obtain a repaired scene image. The scene image data of the radial distortion type is repaired, wherein, ( , (x, y) represents the original position of the distorted point on the scene image data, (x, y) represents the new position after distortion repair, and r is the radius with the center of the optical axis as the origin. The first three terms of the Taylor series expansion are given, and the optical axis center is the point in the image center where no distortion occurs. The scene image data with tangential distortion is subjected to trapezoidal correction, and then processed according to the radial distortion type.

4. A scene-recognition-based intelligent supplemental lighting dynamic control system, characterized in that, The system includes: Complementary lighting module: A directional supplementary lighting device is provided. The initial position of the directional supplementary lighting device must be the same as the direction of the CMOS image acquisition device. The directional supplementary lighting device and the CMOS image acquisition device are at the same horizontal position in space, and the directional supplementary lighting device and the CMOS image acquisition device are fixed on the same base. The viewing angle error between the CMOS image acquisition device and the directional supplementary lighting device is within a preset range. Scene classification module: Constructs scene classifications and adds multiple classification results to the scene dataset; Image acquisition module: Uses the CMOS image acquisition device to acquire the current scene image, obtains scene image data, and judges the scene image data based on the scene dataset; Fill light area module: Obtain the fill light area based on the scene judgment result, and obtain the fill light area position information of the turning fill light device based on the fill light area; Fill light position module: Rotate the steering fill light device to compensate according to the fill light area position information; The system also includes: Scene Dataset Module: The scene dataset module is used to collect scene datasets including front backlighting scenes, side backlighting scenes, opposite lighting scenes, and low-light scenes; Scene feature module: The scene feature module is used to extract scene features from the front backlight scene, the side backlight scene, the opposite light scene and the lack of light scene, and obtain front backlight scene features, side backlight scene features, opposite light scene features and lack of light scene features; Scene dataset adding module: The scene dataset adding module is used to add the front backlight scene features, the side backlight scene features, the opposite light scene features, and the lack of light scene features to the scene dataset; Scene image data segmentation module: The scene image data segmentation module is used to classify the scene image data according to the scene features to obtain problem image regions and non-problem image regions; Problem and non-problem image feature extraction module: performs feature extraction on the problem image region and the non-problem image region respectively to obtain the problem image region features and the non-problem image region features; Problem and non-problem image feature matching module: The problem and non-problem image feature matching module is used to match the features of the problem image region and the features of the non-problem image region with the scene features to obtain the matching result; Fill light region acquisition module: The fill light region acquisition module is used to acquire the fill light region according to the matching result; Matching degree threshold setting module: The matching degree threshold setting module is used to set the matching degree threshold; Matching degree threshold result determination module: The matching degree threshold result determination module is used to determine the matching result according to the matching degree threshold. If the matching result meets the matching degree threshold, the supplementary lighting area is obtained; if not, the scene image data is reclassified, and the features of the problem image area and non-problem image area after classification are re-extracted to obtain updated feature extraction results. Updated fill light region module: The updated fill light region module is used to perform feature matching on the updated feature extraction results and obtain the updated fill light region based on the feature matching results.

Images