Exposure adjustment method, device and storage medium based on motion judgment

Abstract

The present application discloses an exposure adjustment method, device and storage medium based on motion judgment, the method comprising: determining a high-speed area and a low-speed area according to the speed information of each pixel in the image; determining weight information according to the first speed of each pixel in the high-speed area and the second speed of each pixel in the low-speed area, and filtering the brightness information of the image according to the weight information to obtain filtered brightness; determining a gain multiple to be adjusted according to the filtered brightness and a preset target brightness, and determining a target gain multiple according to the gain multiple to be adjusted and a current gain multiple; segmenting a preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain an exposure sub-interval; determining a target exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval; and performing exposure adjustment processing according to the target shutter information and the target gain information corresponding to the target exposure sub-interval. The above scheme can effectively adjust the exposure parameters.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an exposure adjustment method, device and storage medium based on motion judgment. Background Art

[0002] In the current application scenarios of image acquisition devices, if a bright or dark target appears in the picture, most of the existing AE (Auto Exposure) algorithms will adjust the exposure parameters accordingly. Frequent adjustment of exposure parameters will cause the picture to flicker and cause poor shooting effects.

[0003] Especially in night scenes, if the same exposure gain upper limit as that of daytime scenes is used, there will either be serious ghosting problems in the picture or the picture brightness will be insufficient.

[0004] When faced with this type of problem, technicians usually adjust the shutter limit of the image acquisition device after the image acquisition device is installed, which is very time-consuming and labor-intensive. Summary of the invention

[0005] The present application at least provides an exposure adjustment method, apparatus, device and computer-readable storage medium based on motion judgment.

[0006] The first aspect of the present application provides an exposure adjustment method based on motion judgment, comprising: obtaining speed information of each pixel in a current frame image, and determining a high-speed motion area and a low-speed motion area in the current frame image according to the speed information; determining corresponding weight information according to a first speed of each pixel in the high-speed motion area and a second speed of each pixel in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain a filtered brightness of the current frame image; determining a gain multiple to be adjusted according to the filtered brightness and a preset target brightness, and determining a target gain multiple according to the gain multiple to be adjusted and a current gain multiple corresponding to the current frame image; segmenting a preset exposure interval according to an exposure upper limit corresponding to the first speed to obtain a plurality of exposure sub-intervals; determining a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval; and performing exposure adjustment processing according to target shutter information and target gain information corresponding to the target exposure sub-interval.

[0007] In one embodiment, the method of obtaining speed information of each pixel in the current frame image and determining the high-speed motion area and the low-speed motion area in the current frame image based on the speed information includes: obtaining the optical flow speed of each pixel in the current frame image; determining the speed segmentation threshold of the current frame image based on the optical flow speed of each pixel; and segmenting the current frame image into the high-speed motion area and the low-speed motion area based on the speed segmentation threshold and the optical flow speed of each pixel.

[0008] In one embodiment, determining the speed segmentation threshold of the current frame image based on the optical flow speed of each pixel point includes: performing mean calculation based on the optical flow speed of each pixel point to obtain an initial segmentation threshold of the current frame image; performing segmentation processing on the current frame image according to the initial segmentation threshold to obtain a first area and a second area in the current frame image; performing mean calculation on the optical flow speed mean of the first area and the optical flow speed mean of the second area to obtain a current segmentation threshold; in response to the difference between the initial segmentation threshold and the current segmentation threshold being less than a preset threshold difference, determining the current segmentation threshold as the speed segmentation threshold.

[0009] In one embodiment, the corresponding weight information is determined according to the first speed of each pixel in the high-speed motion area and the second speed of each pixel in the low-speed motion area, and the brightness information of the current frame image is filtered according to the weight information to obtain the filtered brightness of the current frame image, including: determining the first speed mean of the high-speed motion area according to the first speed of each pixel in the high-speed motion area, and determining the second speed mean of the low-speed motion area according to the second speed of each pixel in the low-speed motion area; determining the brightness motion weight of each pixel according to the speed information of each pixel, the first speed mean, the second speed mean and a preset motion weight; determining the mean brightness of the current frame image according to the brightness motion weight and the brightness information of the current frame image; determining the brightness filtering weight of the current frame image according to the first speed mean, a preset mean threshold and a preset filtering parameter; inputting the mean brightness and the brightness filtering weight into a preset mean filter for filtering to obtain the filtered brightness.

[0010] In one embodiment, determining the gain multiple to be adjusted based on the filtered brightness and the preset target brightness, and determining the target gain multiple based on the gain multiple to be adjusted and the current gain multiple corresponding to the current frame image, include: determining the gain multiple to be adjusted based on the ratio between the filtered brightness and the preset target brightness; obtaining current shutter information and current gain information corresponding to the current frame image; determining the current gain multiple based on the current shutter information and the current gain information; and summing the gain multiple to be adjusted and the current gain multiple to obtain the target gain multiple.

[0011] In one embodiment, before segmenting the preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain multiple exposure sub-intervals, the method also includes: determining a first speed mean of the high-speed motion area according to the first speed of each pixel point in the high-speed motion area; inputting the first speed mean into a preset mean filter for filtering to obtain a filtering speed; and determining the exposure upper limit according to the filtering speed and a preset speed parameter.

[0012] In one embodiment, the target exposure sub-interval is determined from each exposure sub-interval based on the target gain multiple and the gain information corresponding to each exposure sub-interval, including: determining the segmented gain multiple corresponding to each exposure sub-interval based on the shutter information and gain information corresponding to each exposure sub-interval; determining the target exposure sub-interval based on the target gain multiple and the segmented gain multiple.

[0013] In one embodiment, after determining corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image, the method further includes: determining the brightness multiple of the current frame image according to shutter information and gain information of the current frame image; determining the ambient brightness of the current frame image according to the filtered brightness and the brightness multiple; controlling the working mode of the image acquisition device used to acquire the current frame image according to the ambient brightness and a preset brightness threshold; the working modes include a first mode and a second mode.

[0014] The second aspect of the present application provides an exposure adjustment device based on motion judgment, comprising: an area determination module, used to obtain speed information of each pixel in a current frame image, and determine a high-speed motion area and a low-speed motion area in the current frame image according to the speed information; a brightness filtering module, used to determine corresponding weight information according to a first speed of each pixel in the high-speed motion area and a second speed of each pixel in the low-speed motion area, and filter the brightness information of the current frame image according to the weight information to obtain a filtered brightness of the current frame image; a gain determination module, used to determine a gain multiple to be adjusted according to the filtered brightness and a preset target brightness, and to determine a target gain multiple according to the gain multiple to be adjusted and a current gain multiple corresponding to the current frame image; an interval division module, used to perform interval segmentation processing on a preset exposure interval according to an exposure upper limit corresponding to the first speed to obtain a plurality of exposure sub-intervals; an interval determination module, used to determine a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval; an exposure adjustment module, used to perform exposure adjustment processing according to target shutter information and target gain information corresponding to the target exposure sub-interval.

[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned exposure adjustment method based on motion judgment.

[0016] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned exposure adjustment method based on motion judgment when the program instructions are executed by a processor.

[0017] The above scheme can divide the current frame image into a high-speed motion area and a low-speed motion area according to the speed information by acquiring the speed information of each pixel in the current frame image. According to the first speed of the high-speed motion area and in combination with the second speed of the low-speed motion area, the current frame image can be filtered to obtain the filtered brightness. According to the filtered brightness and the preset target brightness, the target gain multiple that needs to be adjusted can be determined. Then, according to the first speed of the high-speed motion area and the exposure upper limit, the preset exposure interval is segmented to generate exposure sub-intervals. Then, according to the target gain multiple and the gain information of each exposure sub-interval, the target exposure sub-interval can be determined from each exposure sub-interval. In this way, the target shutter information and target gain information corresponding to the target exposure sub-interval can be used as target exposure parameters for exposure adjustment processing, so as to realize exposure adjustment based on motion judgment, avoid the interference of high-speed moving objects on the exposure adjustment process, and optimize the effect of automatic exposure adjustment.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0020] Figure 1 is a flowchart of an exemplary embodiment of the exposure adjustment method based on motion judgment of the present application;

[0021] Figure 2 It is a schematic diagram of a flow chart of controlling a camera to switch between different speed modes in the exposure adjustment method based on motion judgment of the present application;

[0022] Figure 3 is an exemplary mode switching schematic diagram in the exposure adjustment method based on motion judgment of the present application;

[0023] Figure 4 is an exemplary data conversion diagram in the exposure adjustment method based on motion judgment of the present application;

[0024] Figure 5 is a block diagram of an exposure adjustment device based on motion judgment shown in an exemplary embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0026] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0027] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0028] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0029] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0030] For ease of understanding, an exemplary description is now given of one of the applicable scenarios of the exposure adjustment method based on motion judgment of the present application. The technical principle of existing automatic exposure is mainly that when the ambient light brightness changes, the brightness of the raw image data can be kept close to the target brightness by controlling shooting parameters such as gain, exposure time and aperture to avoid overexposure or underexposure.

[0031] However, in some application scenarios, such as image acquisition and image detection on traffic roads, people usually focus on small individual targets such as license plates or objects inside the car. Once large objects such as white or dark vehicles occupy most of the image (affecting the brightness of the image), the existing AE algorithm will frequently adjust the exposure parameters, resulting in flickering and poor shooting effects. Especially at night, if the same exposure gain upper limit as during the day is used, high-speed vehicles in the image will have serious ghosting, and the overall brightness of the image may be insufficient; at this time, the only way is to ask technicians to adjust the shutter upper limit after installing the camera, which is time-consuming, labor-intensive and inefficient.

[0032] It should be noted that in the various embodiments described in the following examples of this application, unless the order of each execution step is clearly stated, or the order of each execution step can be determined according to the execution logic, the order of the execution steps is not limited. Some of the data acquisition steps, data calculation steps, etc. may be acquired in advance or acquired when the data is required, which is not limited here.

[0033] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an exemplary embodiment of the exposure adjustment method based on motion judgment of the present application. Specifically, the following steps may be included:

[0034] Step S110 , obtaining speed information of each pixel in the current frame image, and determining a high-speed motion area and a low-speed motion area in the current frame image according to the speed information.

[0035] Among them, the method of obtaining speed information may include but is not limited to the optical flow method, the frame difference method and the background subtraction method, etc., which are not limited here. Preferably, the optical flow method can be used in this application to obtain the speed information of each pixel in the current frame image. The optical flow method is a method that uses the change of pixels in the image sequence in the time domain and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion information of the object between adjacent frames. For details, please refer to the existing explanation of the optical flow method, which will not be repeated here.

[0036] Furthermore, the image area of ​​the current frame image can be divided according to the speed information of each pixel point, and a high-speed motion area and a low-speed motion area in the current frame image are obtained. Among them, the high-speed motion area refers to an area containing pixels with a higher motion speed, and similarly, the low-speed motion area refers to an area containing pixels with a lower motion speed. For the method of judging whether the speed of a pixel point is higher or lower, the speed threshold (preset or determined in real time) can be compared with the speed information of the pixel point for judgment, which will not be elaborated here.

[0037] Step S120, determining corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image.

[0038] According to the aforementioned steps, it can be known that the speed information between the pixels in the high-speed motion area and the pixels in the low-speed motion area is different. For ease of explanation, this application refers to the speed information of each pixel in the high-speed motion area as the first speed, and the speed information of each pixel in the low-speed motion area as the second speed. It should be noted that the first speed and the second speed in this application are only pseudonyms. In fact, the first speeds of different pixels in the high-speed motion area can be the same or different, and the second speeds of different pixels in the low-speed motion area can also be the same or different, which is not limited here.

[0039] It should be noted that in the implementation process of this application, a variety of preset weight information is pre-set, which may include but is not limited to brightness filter weight, brightness motion weight, etc. Different preset weight information may correspond to different speed information. Therefore, after obtaining the speed information of each pixel point, the corresponding weight information can be determined. Then, the brightness information of the current frame image is filtered according to the obtained weight information to obtain the filtered brightness of the current frame image.

[0040] It is understandable that the RGBY information of each pixel can be determined based on the pixel information of each pixel in the current frame image. The specific method can refer to the existing related technology and will not be described here. Among them, R refers to red, G refers to green, B refers to blue, and Y refers to brightness. By counting the RGBY information of all pixels in the current frame image, the RGBY statistical value can be obtained ( ), This is equivalent to the brightness information of the current frame image. By filtering it in combination with the acquired weight information, the filtered brightness of the current frame image can be obtained.

[0041] Step S130, determining the gain multiple to be adjusted according to the filtered brightness and the preset target brightness, and determining the target gain multiple according to the gain multiple to be adjusted and the current gain multiple corresponding to the current frame image.

[0042] The filter brightness is equivalent to the brightness of the current frame image, and the preset target brightness is equivalent to the desired target brightness (that is, the brightness that needs to be adjusted). Therefore, the gain multiple to be adjusted in the process of adjusting from the current brightness to the target brightness can be determined by the current brightness and the target brightness. , the unit of gain can be db.

[0043] It should be noted that during the image acquisition process, the camera (image sensor) can usually obtain the currently set gain multiple (current gain multiple curr_db). Therefore, by adding the gain multiple to be adjusted on the basis of the current gain multiple, the target gain multiple aim_db can be determined, and its mathematical expression can be:

[0044]

[0045] Step S140, segmenting the preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain a plurality of exposure sub-intervals.

[0046] It should be noted that, for each camera (image sensor), a corresponding exposure interval is usually pre-set. The left and right endpoints of the exposure interval respectively represent the lower limit vmin and upper limit vmax of the exposure time in the interval. vmax can be the lower limit number of exposure rows of the image sensor in non-long exposure mode (for example, 1 line), and vmax can be the upper limit number of exposure rows of the image sensor in non-long exposure mode, which is also equivalent to the upper and lower limits of its shutter, and will not be elaborated here.

[0047] Exemplarily, for the sake of convenience, the exposure time is mainly characterized by the number of exposure rows as an example in this application. For example, the preset exposure interval can be [1, vmax], which represents that the number of exposure rows of the camera can be adjusted between the lower exposure limit of 1 row and the upper exposure limit of vmax row, thereby adjusting the exposure effect of the image. In the exposure adjustment process, the exposure adjustment strategy is usually: the number of exposure rows of the camera can be adjusted from 1 row to vmax row (which can be recorded as exposure adjustment interval segment 1). If the exposure effect of the image still needs to be adjusted after the number of exposure rows of the camera reaches vmax, the exposure adjustment can be achieved by changing the gain (which can be recorded as exposure adjustment interval segment 2). Among them, a corresponding preset gain interval can also be pre-set in the same way, such as [0, 100], which is equivalent to representing that after the number of exposure rows of the camera reaches vmax, the gain of the camera can be adjusted from 0db to 100db, thereby achieving further exposure adjustment.

[0048] The above method can effectively adjust the exposure effect, but since the upper limit of exposure is raised, it is difficult to avoid the problem of motion smear of high-speed moving objects in the image. Therefore, the present application can determine the corresponding upper limit of exposure according to the first speed corresponding to the high-speed moving area in the image. Then, the preset exposure interval is segmented according to the exposure upper limit to obtain multiple exposure sub-intervals. Then, the exposure is adjusted step by step based on the shutter and gain corresponding to each exposure sub-interval, which can effectively reduce the interference of high-speed moving objects on exposure. The brightness exposure upper limit and the camera isp (Image Signal Processor) parameters can be linked to effectively suppress the problem of motion smear.

[0049] Step S150: determining a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval.

[0050] Exemplarily, the above steps are combined for explanation. For example, the preset exposure interval is [1, vmax]. This is equivalent to adjusting the number of exposure rows from 1 to vmax in the process of improving the exposure effect, and the gain information of the camera can remain unchanged at 0db (such as exposure adjustment interval segment 1 in the above example); if the exposure effect still needs to be improved after the number of exposure rows reaches vmax, the gain can be adjusted from 0db to 100db (such as exposure adjustment interval segment 2 in the above example).

[0051] Furthermore, after obtaining the exposure upper limit corresponding to the high-speed motion area Afterwards, the preset exposure interval can be divided into exposure sub-intervals [1, ]and[ , vmax]. In the process of adjusting exposure according to the exposure sub-interval, the number of exposure lines can be adjusted from 1 line to , and keep the gain at 0db (get the new exposure adjustment interval 1, this is to change the exposure effect by adjusting the number of exposure rows). If you need to continue to improve the exposure effect, keep the number of exposure rows at , adjust the exposure gain from 0db to 100db (get new exposure adjustment interval 2, this is to change the exposure effect by adjusting the exposure gain), so as to avoid the problem of smearing of high-speed moving objects caused by the continuous increase in the number of exposure lines. If the exposure gain is 100db and the exposure effect still needs to be improved, keep the exposure gain at 100db and increase the number of exposure lines from The row is adjusted to the vmax row (exposure adjustment interval segment 3 is obtained, which is a change in exposure effect achieved by further adjusting the number of exposure rows).

[0052] It should be noted that, in the present application, when the number of exposure lines is adjusted and / or the exposure gain is adjusted, it can be adjusted once or gradually according to the adjustment step length or according to a preset linear relationship, which is not limited here. In addition, the above embodiment is for the convenience of explanation, so the above exposure interval and gain interval are used for simple examples. In actual application scenarios, more preset exposure intervals and / or preset gain intervals can be included, which can be adjusted as needed, and will not be described in detail here.

[0053] It can be seen from the above embodiment that each exposure sub-interval also has corresponding left and right endpoints. Taking the right endpoint of each exposure sub-interval (equivalent to the maximum value of each exposure sub-interval) as an example, the gain information equivalent multiple equal_db corresponding to each exposure adjustment interval segment can be determined according to the shutter shut (in units of lines) and gain gain (in units of db) corresponding to the right endpoint. The mathematical expression thereof can be:

[0054]

[0055] The base number in the logarithm calculation may be 10, which is a simple expression and will not be described in detail later.

[0056] Therefore, the gain information equal_db corresponding to each exposure adjustment interval segment can be traversed in order from small to large. In response to a certain equal_db being greater than aim_db, the exposure adjustment interval segment corresponding to the current equal_db meets the aim_db exposure requirement. It should be noted that the increase in camera gain may increase the noise in the image, thereby affecting the image quality. Therefore, although there may be one or more exposure adjustment interval segments where equal_db is greater than aim_db, the exposure adjustment interval segment closest to aim_db (i.e., the interval segment when traversing to equal_db is greater than aim_db) can be determined by the above-mentioned sequential traversal method, thereby obtaining an image with better exposure effect and better image quality. However, in actual application scenarios, it is also possible to select a target exposure sub-interval (which may affect the image quality) in other exposure adjustment interval segments where equal_db is greater than aim_db, which is not limited here.

[0057] Step S160 , performing exposure adjustment processing according to the target shutter information and the target gain information corresponding to the target exposure sub-interval.

[0058] Combined with the above steps, according to The exposure adjustment interval segment can determine the target shutter information and target gain information corresponding to the sub-interval.

[0059] For example, the exposure sub-interval in the above embodiment is still used as an example for description. ]and[ , vmax], and in a scene where the gain interval is [0,100], if aim_db is in exposure adjustment interval 2, it can be determined that the target shutter information aim_exp corresponding to aim_db should be fixed to Then, according to aim_db and the gain information equivalent multiple of the right endpoint of the adjacent previous exposure adjustment interval (exposure adjustment interval 1), Then the target gain information aim_gain that needs to be adjusted currently can be determined.

[0060] In summary, by setting the target shutter information aim_exp and the target gain information aim_gain to the corresponding camera (image sensor sensor), the exposure adjustment can be completed so that the subsequently collected images can meet the exposure requirements of aim_db.

[0061] It can be seen that the present application can divide the current frame image into a high-speed motion area and a low-speed motion area according to the speed information by obtaining the speed information of each pixel in the current frame image. According to the first speed of the high-speed motion area and combined with the second speed of the low-speed motion area, the current frame image can be filtered to obtain the filtered brightness. The target gain multiple that needs to be adjusted can be determined according to the filtered brightness and the preset target brightness. Then, according to the first speed of the high-speed motion area combined with the exposure upper limit, the preset exposure interval is segmented to generate exposure sub-intervals. Then, according to the target gain multiple and the gain information of each exposure sub-interval, the target exposure sub-interval can be determined from each exposure sub-interval. In this way, the target shutter information and target gain information corresponding to the target exposure sub-interval can be used as target exposure parameters for exposure adjustment processing, so as to realize exposure adjustment based on motion judgment, avoid the interference of high-speed moving objects on the exposure adjustment process, and optimize the effect of automatic exposure adjustment.

[0062] Based on the above embodiments, the present embodiment of the present application describes the steps of obtaining the speed information of each pixel in the current frame image and determining the high-speed motion area and the low-speed motion area in the current frame image according to the speed information. Specifically, the method of this embodiment includes the following steps:

[0063] Obtain the optical flow velocity of each pixel in the current frame image; determine the speed segmentation threshold of the current frame image according to the optical flow velocity of each pixel; and segment the current frame image into a high-speed motion area and a low-speed motion area according to the speed segmentation threshold and the optical flow velocity of each pixel.

[0064] For example, the speed information of each pixel in the present application may be the optical flow speed flow_u calculated according to the optical flow method. Then, the high-speed motion area and the low-speed motion area in the current frame image are segmented according to the optical flow speed of each pixel and the speed segmentation threshold. Among them, the speed segmentation threshold may be preset or determined according to the optical flow speed of the current frame image.

[0065] For example, in this application, a global threshold method may be used to determine the corresponding segmentation threshold T according to the optical flow velocity flow_u obtained in the global motion calculation process. In addition, other threshold segmentation methods (Otsu method, percentage threshold method, adaptive threshold segmentation, iterative threshold segmentation, etc.) may also be used, which are not limited here.

[0066] According to the optical flow speed of all pixels (or part of the pixels) in the current frame image and the number of these pixels, the average optical flow speed of the current frame image is determined. In addition, it can also be other mathematical statistical values ​​such as median and weighted value, which are not limited here. Then the speed segmentation threshold of the current frame image can be determined according to the average optical flow speed of the current frame image. Then the current frame image is divided into a high-speed motion area and a low-speed motion area according to the speed segmentation threshold. Among them, the area where the pixels in the current frame image whose optical flow speed is greater than the speed segmentation threshold is located can be determined as the high-speed motion area, and the area where the pixels in the current frame image whose optical flow speed is less than or equal to the speed segmentation threshold is located can be determined as the low-speed motion area.

[0067] Based on the above embodiment, the embodiment of the present application describes the steps of determining the speed segmentation threshold of the current frame image according to the optical flow speed of each pixel point. Specifically, the method of this embodiment includes the following steps:

[0068] The initial segmentation threshold of the current frame image is obtained by calculating the mean value of the optical flow velocity of each pixel point; the current frame image is segmented according to the initial segmentation threshold to obtain the first area and the second area in the current frame image; the mean value of the optical flow velocity of the first area and the mean value of the optical flow velocity of the second area are calculated to obtain the current segmentation threshold; in response to the difference between the initial segmentation threshold and the current segmentation threshold being less than the preset threshold difference, the current segmentation threshold is determined as the speed segmentation threshold.

[0069] In combination with the above-mentioned embodiment, when determining the speed segmentation threshold of the current frame image according to the optical flow speed of the pixel point in the present application, the specific method may include but is not limited to: determining the initial segmentation threshold of the current frame image according to the average optical flow speed of the current frame image First, according to the initial segmentation threshold The current frame image is divided into two types of images: the initial high-speed area (first area) and the initial low-speed area (second area). Similarly, the optical flow velocity mean m1 is determined based on the optical flow velocity of each pixel in the first area and the number of pixels in the first area, and the optical flow velocity mean m2 is determined based on the optical flow velocity of each pixel in the second area and the number of pixels in the second area. The average value of the optical flow velocity mean of the two area images is calculated based on m1 and m2, which is the current segmentation threshold. , its mathematical expression can be:

[0070]

[0071] If the current segmentation threshold and the initial segmentation threshold If the difference between them is less than the preset threshold difference, the current segmentation threshold The speed segmentation threshold is determined, and the current frame image is segmented into a high-speed motion area and a low-speed motion area based on it.

[0072] If the current segmentation threshold and the initial segmentation threshold If the difference between them is greater than or equal to the preset threshold difference, the current segmentation threshold can be used as the initial segmentation threshold for the next iterative calculation according to the principle of iterative calculation. (equivalent to Assign to , so that = ). Referring to the previous steps, similarly, according to the current initial segmentation threshold (The actual value is ), re-segment the current frame image to obtain the newly segmented first region and second region, and similarly calculate the current segmentation threshold according to the mean m1 of the first region and the mean m2 of the second region in the current state , and continue to calculate the current With the current A comparison is performed to determine whether to determine the speed segmentation threshold, or the above operation is repeated iteratively until the difference between the two adjacent thresholds is less than the preset threshold difference, and the finally calculated threshold can be determined as the speed segmentation threshold T.

[0073] Based on the above embodiment, the embodiment of the present application describes the steps of determining corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image. Specifically, the method of this embodiment includes the following steps:

[0074] A first speed mean of the high-speed motion area is determined according to the first speed of each pixel in the high-speed motion area, and a second speed mean of the low-speed motion area is determined according to the second speed of each pixel in the low-speed motion area; a brightness motion weight of each pixel is determined according to the speed information of each pixel, the first speed mean, the second speed mean and the preset motion weight; a mean brightness of the current frame image is determined according to the brightness motion weight and the brightness information of the current frame image; a brightness filtering weight of the current frame image is determined according to the first speed mean, the preset mean threshold and the preset filtering parameter; the mean brightness and the brightness filtering weight are input into a preset mean filter for filtering to obtain the filtered brightness.

[0075] In combination with the above-mentioned embodiment, the method according to the above-mentioned embodiment can determine the high-speed motion area and the low-speed motion area in the current frame image by determining the speed segmentation threshold. Among them, the speed information of each pixel point in the high-speed motion area is generally referred to as the first speed, and the speed information of each pixel point in the low-speed motion area is generally referred to as the second speed. Therefore, the first speed average of the high-speed motion area can be determined by the first speed of each pixel point in the high-speed motion area. , and determining the second velocity mean of the low-speed motion area by the second velocity of each pixel in the low-speed motion area .

[0076] Preset motion weights are pre-set in this application and .in, It can be an empirical constant determined based on test results and specific application scenarios. Less than . For example, <1.0, the lower the value, the smaller the brightness motion weight representing the high-speed motion area. It can be set to 1.0, with no specific limit.

[0077] For example, each pixel in the current frame image can be traversed. If the optical flow velocity flow_u of the current pixel is greater than , then we can Determine the brightness motion weight of the current pixel ; On the contrary, if the optical flow velocity flow_u of the current pixel is less than , then you can Determine the brightness motion weight of the current pixel ; If the optical flow velocity flow_u of the current pixel is and In between, you can and Perform weighted summation to obtain the brightness motion weight Therefore, its mathematical expression can be:

[0078]

[0079] in, is used to and The weight value for weighted summation may be a preset weight, or may be determined based on the speed information of the current pixel, the first speed average, and the second speed average, which is not limited here. For example, its mathematical expression may be:

[0080]

[0081] Furthermore, the brightness motion weight of all or part of the pixels can be used. And the brightness information of the current frame image Determine the average brightness avg_y of the current frame image, which can be mathematically expressed as:

[0082]

[0083] In addition, in this application, according to the first speed average , a preset mean threshold and a preset filtering parameter to determine the brightness filtering weight of the current frame image, can be similarly referred to the description in the above embodiment. In the specific implementation process, a preset mean threshold can be preset. and , and preset filter parameters and . and It can be used to evaluate the size of the first velocity mean in the high-speed motion area, and then select the corresponding preset filter parameter as the brightness filter weight .

[0084] Exemplarily, the brightness filter weight is determined The mathematical expression of can be:

[0085]

[0086] in, is used to and The weight value for weighted summation may be a preset weight, or may be determined based on the first speed mean and a preset mean threshold, which is not limited here. For example, its mathematical expression may be:

[0087]

[0088] In summary, the mean brightness avg_y and the brightness filter weight Input the preset mean filter for filtering, and then you can get the filtered brightness output by the mean filter. , its mathematical expression can be:

[0089]

[0090] Based on the above embodiment, the embodiment of the present application describes the steps of determining the gain multiple to be adjusted according to the filter brightness and the preset target brightness, and determining the target gain multiple according to the gain multiple to be adjusted and the current gain multiple corresponding to the current frame image. Specifically, the method of this embodiment includes the following steps:

[0091] The gain multiple to be adjusted is determined according to the ratio between the filtered brightness and the preset target brightness; the current shutter information and the current gain information corresponding to the current frame image are obtained; the current gain multiple is determined according to the current shutter information and the current gain information; the gain multiple to be adjusted and the current gain multiple are summed to obtain the target gain multiple.

[0092] Combined with the above embodiment, the filter brightness can be obtained from the above embodiment. , combined with the preset target brightness , then the gain multiple adj_db to be adjusted can be determined, and its mathematical expression can be:

[0093]

[0094] Optionally, in order to optimize the image display quality and prevent the exposure adjustment process from oscillating back and forth, a gradual adjustment can be performed during the exposure adjustment process. For example, the gain multiple adj_db to be adjusted that is initially determined each time is reduced to obtain the reduced gain multiple adj_db to be adjusted, and the adjustment is made based on this. For example, the adjustment can be made based on one-fifth of the gain multiple adj_db to be adjusted that is calculated each time, and its mathematical expression can be:

[0095]

[0096] Then, the current shutter information corresponding to the current frame image can be obtained and current gain information There are many ways to obtain data (from the image information contained in the current frame image, and / or from the sensor data, etc.), which will not be described here. The current gain multiple curr_db is calculated, and its mathematical expression can be:

[0097]

[0098] In summary, we can and Determine the target gain multiplier aim_db.

[0099] On the basis of the above embodiment, the present embodiment of the present application describes the steps before the preset exposure interval is segmented according to the exposure upper limit corresponding to the first speed to obtain multiple exposure sub-intervals. Specifically, the method of this embodiment includes the following steps:

[0100] The first speed mean of the high-speed motion area is determined according to the first speed of each pixel point in the high-speed motion area; the first speed mean is input into a preset mean filter for filtering to obtain a filtering speed; and the exposure upper limit is determined according to the filtering speed and the preset speed parameter.

[0101] The preset speed parameter may be a preset empirical constant a, which refers to the maximum displacement that a moving object in the current scene is allowed to make within the shutter time, or may be set as a reference based on the maximum displacement, which will not be elaborated here.

[0102] Combined with the above-mentioned embodiment, the first speed average value of each pixel point in the high-speed motion area can be determined according to the first speed of each pixel point. . Input it into the preset mean filter for filtering, and the filtering speed of the mean filter output can be obtained. .

[0103] Furthermore, according to the filtering speed and preset speed parameter a, the current shutter upper limit (exposure upper limit) can be determined , its mathematical expression can be:

[0104]

[0105] Therefore, referring to the description of the above embodiment, if the exposure lower limit threshold of the exposure interval [1, vmax] is 1 and the exposure upper limit threshold is vmax, the interval can be segmented to obtain several exposure sub-intervals, such as [1, ]as well as[ , vmax].

[0106] Based on the above embodiment, the embodiment of the present application describes the steps of determining the target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval. Specifically, the method of this embodiment includes the following steps:

[0107] The segmented gain multiple corresponding to each exposure sub-interval is determined according to the shutter information and gain information corresponding to each exposure sub-interval; and the target exposure sub-interval is determined according to the target gain multiple and the segmented gain multiple.

[0108] In conjunction with the above embodiment, each exposure sub-interval may have its corresponding shutter information and gain information, which may be determined according to the shutter information and gain information corresponding to the preset exposure interval. For example, in the above embodiment, it is illustrated that the shutter information of the first exposure sub-interval is [1, ], the gain information can be 0db; the shutter information of the exposure sub-interval of the second segment is , the gain information can be [0,100]; the shutter information of the exposure sub-interval of the third segment is [ , vmax], the gain information can be 100db. According to the vertex of each interval (the right endpoint, equivalent to the maximum value of the shutter information shut and the maximum value of the gain information gain), the segmented gain multiple equal_db of each interval can be determined. The sub-interval when equal_db is greater than aim_db can be determined as the target exposure sub-interval, and its final target shutter aim_exp and target gain aim_gain are set to the sensor.

[0109] Based on the above embodiment, the embodiment of the present application describes the steps after determining the corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image. Specifically, the method of this embodiment includes the following steps:

[0110] Determine the brightness multiple of the current frame image according to the shutter information and gain information of the current frame image; determine the ambient brightness of the current frame image according to the filtered brightness and the brightness multiple; control the working mode of the image acquisition device used to acquire the current frame image according to the ambient brightness and a preset brightness threshold; the working mode includes a first mode and a second mode.

[0111] It should be noted that in the specific application process of this embodiment, the working mode of the image acquisition device (such as a camera, a video camera, etc., which is not limited here) can also be adjusted. Among them, the working mode of the image acquisition device of the present application can include but is not limited to the first mode and the second mode.

[0112] Taking the camera as an example, the first mode refers to a mode suitable for the camera to operate in an environment with sufficient brightness (when the ambient brightness is greater than the preset brightness threshold, it can be considered that the brightness is sufficient, and it can also be called the day mode for ease of understanding), and the second mode refers to a mode suitable for the camera to operate in an environment with insufficient brightness (when the ambient brightness is less than or equal to the preset brightness threshold, it can be considered that the brightness is insufficient, and it can also be called the night mode for ease of understanding). A fill light may also be provided in the current scene, and there may or may not be a communication connection between the camera and the fill light. The two may be integrally or separately arranged, which is not limited here. For example, in the day mode, the fill light may be in a dormant state and not provide fill light for the camera; while in the night mode, the fill light may be activated to provide fill light for the camera.

[0113] For example, the fill light of the present application may include but is not limited to an infrared fill light, a white light fill light and a mixed fill light. Taking the application scenario of the infrared fill light as an example, in the day mode, the camera can normally capture color images; while in the night mode, the camera captures grayscale images.

[0114] As another example, it should be noted that the camera's working parameters in day mode and night mode may be the same or different. For example, when the camera is working in day mode, due to sufficient brightness, the camera's operating parameters such as aperture, shutter, iso and gain may be smaller than when the camera is working in night mode to avoid overexposure during the day. Therefore, when controlling the camera to work in different working modes, in addition to controlling the corresponding fill light, the corresponding working parameters can also be adjusted, which will not be elaborated here.

[0115] In this embodiment, the brightness multiple X of the current frame image can be determined according to the shutter information shut and the gain information gain of the current frame image, and its mathematical expression can be:

[0116]

[0117] Furthermore, according to the filter brightness The ambient brightness lum of the current frame image can be determined by the brightness multiple X, and its mathematical expression can be:

[0118]

[0119] It should also be noted that if the image acquisition device uses an automatic aperture lens, since the aperture size will also affect the imaging exposure effect of the image, the equivalent db number of the current aperture can also be approximately determined as the ambient brightness through a pre-calibration method or the relationship between the lens step length and the light-passing area, which will not be elaborated here.

[0120] Thus, the ambient brightness lum can be compared with a preset brightness threshold (there can be one or more preset brightness thresholds, which are not limited here) to determine whether the current ambient brightness is sufficient or insufficient. Among them, if the current frame image is greater than the preset brightness threshold, it is considered that the brightness is sufficient; if the current frame image is less than or equal to the preset brightness threshold, it is considered that the brightness is insufficient.

[0121] Optionally, if the lum of consecutive or non - consecutive n frame images collected by the image acquisition device is greater than the preset brightness threshold (or the lum of a certain proportion of the images is greater than the preset brightness threshold, which can be compared with a preset image quantity threshold), it can be determined that the brightness is sufficient. If the lum of consecutive or non - consecutive n frame images collected by it is less than or equal to the preset brightness threshold (or the lum of a certain proportion of the images is less than or equal to the preset brightness threshold), it can be determined that the brightness is insufficient. Among them, if the brightness is sufficient, the camera is controlled to work in the daytime mode; if the brightness is insufficient, the camera is controlled to work in the night mode.

[0122] Optionally, the preset brightness threshold can also be b1 and b2 (b1 < b2, and the specific values are not limited). If the lum of consecutive or non - consecutive n frame images collected by the image acquisition device is greater than b2 (or the lum of a certain proportion of the images is greater than b2, which can be compared with a preset image quantity threshold), it can be determined that the brightness is sufficient. If the lum of consecutive or non - consecutive n frame images collected by it is less than b1 (or the lum of a certain proportion of the images is less than b1), it can be determined that the brightness is insufficient.

[0123] Taking an example based on one of the above - mentioned examples (which can be adjusted adaptively in actual application scenarios), if the camera is currently in the night mode and the lum of consecutive n frame images is greater than b2, the camera can be controlled to switch to the daytime mode; otherwise, the night mode is continued, and the counted n - frame quantity is cleared to 0, and the quantity counting is restarted. If the camera is currently in the daytime mode and the lum of consecutive n frame images is less than b1, the camera can be controlled to switch to the night mode; otherwise, the daytime mode is continued, and the counted n - frame quantity is cleared to 0, and the quantity counting is restarted.

[0124] Based on the above - mentioned embodiments, the embodiments of the present application can also provide a method for calculating the actual shutter upper limit in combination with the control of the camera and the fill light.

[0125] Exemplarily, first, it can be determined whether the fill light is a controllable fill light (a fill light that can be controlled), which can be preset or obtained in real - time from the attribute information of the fill light, and this is not elaborated here. If so, an empirical constant c can be preset, the pulse width of the fill light is set to t, and the current shutter upper limit of the camera is set to .

[0126] In some scenarios (such as road detection scenarios), the brightness of ambient light (usually road street light fill light) is weak, and because the vehicle license plate is placed vertically and will reflect light, it is difficult to be sensed by the camera sensor, so the main fill light of the vehicle comes from the strobe fill light. Therefore, it can be approximately considered that the fill light time of the license plate is controlled by the pulse width of the strobe light (the pulse width of the strobe light is generally less than the shutter upper limit), so that the camera shutter upper limit can be controlled to increase appropriately to increase the overall image brightness without causing the license plate to have a ghosting phenomenon. The increase ratio is the empirical constant c, but if c is too large, it will still cause vehicle body ghosting, so c can be set within a certain range based on experience (determined according to the actual application scenario, which will not be elaborated here), for example, c can generally take a value range of 1≤c≤1.5. Therefore, the mathematical expression of the actual shutter upper limit of the camera can be:

[0127]

[0128] If the fill light is not a controllable fill light, the shutter upper limit determination process of the above embodiment may not be performed.

[0129] Based on the above embodiment, this embodiment can also provide a method to filter the high-speed motion area according to the filtering speed. and preset empirical constants , Dynamically adjust 3DNR temporal and spatial intensity parameters.

[0130] In order to solve the problem of smearing, the time domain parameters of 3dnr need to be reduced, and the spatial domain parameters need to be increased. The mathematical expression can be:

[0131]

[0132] in and This is the base value in the configuration.

[0133] On the basis of the above embodiments, this embodiment can also provide a method for controlling the working mode of the camera. The working modes in this embodiment may include but are not limited to high-speed mode, medium-speed mode and low-speed mode. Among them, high-speed mode refers to the working mode adopted by the camera when shooting high-speed moving objects, medium-speed mode refers to the working mode adopted by the camera when shooting medium-speed moving objects, and medium-speed mode refers to the working mode adopted by the camera when shooting low-speed moving objects. The judgment method of high speed, medium speed and low speed can still be compared and judged by pre-setting thresholds. It should be noted that the high-speed mode, medium-speed mode and low-speed mode in this embodiment do not conflict with the daytime mode and night mode in the above embodiments. Both can be set in the same camera at the same time, and will not be repeated here. For example, the shutter speed of the camera in the high-speed mode is higher than the shutter speed in the medium-speed mode, and the shutter speed of the camera in the medium-speed mode is higher than the shutter speed in the low-speed mode. In this way, the camera can avoid the phenomenon of ghosting as much as possible when shooting high-speed moving objects.

[0134] For example, reference may be made to Figure 2 As shown, Figure 2 1 is a flow chart of controlling the camera to switch between different speed modes in the exposure adjustment method based on motion judgment of the present application. In the specific application process of this embodiment, its implementation steps may include but are not limited to:

[0135] Step S1: Determine whether the current mode of the camera is the low-speed mode; if so, execute step S2; if not, execute step S4.

[0136] Step S2: Determination Is it greater than the preset threshold value L2? If so, the camera continues to have the frame number M2++ in this mode, and executes step S3; if not, the mode continuous frame numbers M1 and M2 are cleared to 0, and the subsequent judgment process ends.

[0137] Step S3: Determine whether the mode duration frame number M2 is greater than the preset empirical constant K; if so, set the working mode to medium-speed mode, switch the overall isp configuration of the camera to the corresponding medium-speed configuration, and then clear M1 and M2 to 0, and end; if not, end directly.

[0138] Step S4: Determine whether the current mode is the medium speed mode; if so, proceed to step S5; if not, proceed to step S10.

[0139] Step S5: Determination Is it greater than the preset threshold L4? If so, the camera clears the continuous frame number M1 in this mode to 0, M2++, and enters step S6; if not, enters step S7.

[0140] Step S6: Determine whether the mode duration frame number M2 is greater than the empirical constant K; if so, set the camera working mode to high-speed mode, switch the overall isp configuration to high-speed configuration, and clear M1 and M2 to 0, and end; if not, end the subsequent judgment process.

[0141] Step S7: Determination Is it less than the preset threshold L1? If so, continue the frame number M1++, M2 is cleared to 0, and enter step S8; if not, M1 and M2 are cleared to 0.

[0142] Step S8: Determine whether the continuous frame number M1 is greater than the empirical constant K; if so, set the mode to low-speed mode, and switch the overall isp configuration to low-speed configuration, clear M1 and M2 to 0, and end; if not, end.

[0143] Step S9: Determination Is it greater than the preset threshold L3? If so, continue the number of frames M1++ and enter step S10; if not, clear M1 and M2 to 0 and end.

[0144] Step S10: Check whether the continuous frame number M1 is greater than the empirical constant K; if so, set the mode to medium-speed mode, and switch the overall isp configuration to medium-speed configuration, clear M1 and M2 to 0, and end; if not, end.

[0145] Among them, L4≥L3≥L2≥L1, the mode adjustment method from step S1 to step S10 can refer to the following Figure 3 As shown, Figure 3 It is an exemplary mode switching diagram in the exposure adjustment method based on motion judgment of the present application.

[0146] On the basis of the above-mentioned embodiment, this embodiment can also provide a method for obtaining the optical flow velocity of a pixel. Exemplarily, in the specific implementation process, the raw data of two adjacent frames of images can be used to calculate the global motion information of each pixel between the two frames of images through the pyramid optical flow method; or the statistical information calculated by ISP can be directly used (the information calculated by ISP is equivalent to the information that has been downsampled), and the motion information can be determined by the optical flow method without performing multi-layer pyramid calculations to achieve downsampling, which can greatly reduce the amount of calculation, but the accuracy of the result will be slightly reduced. It can be determined specifically according to actual needs and the specifications of the equipment running this method (parameters such as computing power), which will not be elaborated here.

[0147] Taking raw data as an example, the steps of this embodiment may include but are not limited to:

[0148] Step S11, obtaining raw data of two adjacent frames of images and corresponding exposure information.

[0149] Step S12, calculate the multiples X1 and X2 of the exposure values ​​of the previous and next two frames and the preset exposure value. For example, the shutter speed of the current video frame is (unit is row), the gain is (Unit: db), the shutter speed corresponding to the preset exposure value is , the gain is , then the mathematical expression of the multiple X can be:

[0150]

[0151] It should be noted that if the camera uses an automatic aperture lens, the equivalent db number of the current aperture can also be obtained by pre-calibration or by approximate calculation based on the relationship between the lens step length and the light-passing area.

[0152] Step S13, convert the two frames of raw data into statistical data respectively to obtain , Four components; data conversion can refer to the prior art, which will not be described here. For example, it can be as follows Figure 4 As shown, Figure 4 This is an exemplary data conversion diagram in the exposure adjustment method based on motion judgment of this application. The raw data is a group of data represented by every 4 points, so Figure 4 Every four points in the , , the length and width of the resolution of the final statistical value are divided by 2 respectively. Its mathematical expression can be:

[0153]

[0154] Among them, BLC_R, BLC_Gr, BLC_Gb, and BLC_B are the black level values ​​corresponding to the raw data, which can generally be obtained through prior calibration and will not be described in detail here.

[0155] Step S14, divide the Y component in the statistical information of the raw data of the previous and next two frames by the calculated multiples X1 and X2 respectively, and then obtain the raw statistical information of the previous and next two frames at the same exposure level. and , .

[0156] Step S15: Statistical information of the raw images of the two frames before and after at the same exposure level and The data is reduced m times according to the ratio k, and m layers of pyramid data are obtained respectively. Among them, k and m are both empirical values, which can be set according to the resolution of the original statistical information, and will not be described here.

[0157] Step S16, traverse all layers of the pyramid and collect statistics on the nth layer and Perform mean filtering to reduce noise and obtain filtered statistical information Among them, the mean filtering process can use the convolution function, which will not be described here.

[0158] Step S17, using a preset gradient function to filter the statistical information Extract the x and y direction gradients respectively and get the x direction gradient , the gradient in the y direction , and obtain the averaged x-direction gradient and the y-direction gradient ; Its mathematical expression can be:

[0159]

[0160] Step S18: The filtered statistical information Subtract to get the frame difference , its mathematical expression can be:

[0161]

[0162] Step S19, determine whether the currently traversed pyramid level is at the highest level of the pyramid; if so, the optical flow speed in the x and y directions of the current level The matrix is ​​initialized to all 0s; if not, the matrix calculated in the previous layer is used. The matrix is ​​multiplied by the pyramid scaling factor k as the current layer The initial value of the matrix. Its mathematical expression can be:

[0163]

[0164] Step S20: determine whether the current number of iterations i exceeds the iteration number threshold (empirical constant) or whether the difference delta calculated before and after is less than the preset threshold (an empirical constant that may vary with the resolution of the current pyramid); if so, exit the iteration and proceed to step S25; if not, proceed to step S21.

[0165] Step S21, determine whether the current layer is the first iteration of the pyramid; if so, keep The matrix remains unchanged; if not, the result of the previous iteration is Assign the matrix to .

[0166] Step S22, the optical flow speed in the x and y directions of the current layer The matrix is ​​mean filtered to obtain the filtered matrix.

[0167] Step S23, based on the data matrix calculated above, use the local average and optical flow constraint method to calculate the optical flow speed in the x and y directions , its mathematical expression can be:

[0168]

[0169]

[0170]

[0171] in, It is a preset compensation constant and cannot be 0.

[0172] Step S24, calculate the difference delta before and after, which can be expressed mathematically as:

[0173]

[0174] Then, the number of iterations i is increased by 1, and the process returns to step S20 for iteration.

[0175] Step S25, until all pyramids have been calculated, use the optical flow speed in the x and y directions of the bottom pyramid result , the directionless optical flow velocity flow_u is calculated. Its mathematical expression can be:

[0176]

[0177] In summary, the present application can use the optical flow method to calculate the speed information of each pixel based on the raw data of the previous and next two frames, and effectively judge the motion area to divide the image into high-speed and low-speed motion areas. The average motion of the high-speed motion area is used to generate the current frame filter weight; then combined with the average motion of the low-speed motion area, the brightness motion weight of each pixel is generated, and the brightness of the current frame is calculated together. According to the average motion of the high-speed motion area combined with the exposure upper limit, the ISP parameters are dynamically adjusted, and new exposure segments are regenerated to determine the final exposure parameters. The present application can effectively reduce the interference of high-speed moving objects on exposure; combining the brightness exposure upper limit and the ISP parameters can effectively suppress the problem of motion smear. Among them, you can also choose to use the statistical information obtained by the ISP statistics without performing multi-layer pyramid calculations, which can greatly reduce the amount of calculations.

[0178] It should be further explained that the execution subject of the exposure adjustment method based on motion judgment may be an exposure adjustment device based on motion judgment, for example, the exposure adjustment method based on motion judgment may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the exposure adjustment method based on motion judgment may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0179] Figure 5 FIG. 1 is a block diagram of an exposure adjustment device based on motion judgment, which is shown in an exemplary embodiment of the present application. Figure 5 As shown, the exemplary exposure adjustment device 500 based on motion judgment includes: a region determination module 510, a brightness filtering module 520, a gain determination module 530, an interval division module 540, an interval determination module 550 and an exposure adjustment module 560. Specifically:

[0180] The region determination module 510 is used to obtain speed information of each pixel in the current frame image, and determine the high-speed motion region and the low-speed motion region in the current frame image according to the speed information.

[0181] The brightness filtering module 520 is used to determine the corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filter the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image.

[0182] The gain determination module 530 is used to determine the gain multiple to be adjusted according to the filtered brightness and the preset target brightness, and to determine the target gain multiple according to the gain multiple to be adjusted and the current gain multiple corresponding to the current frame image.

[0183] The interval division module 540 is used to perform interval segmentation processing on the preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain multiple exposure sub-intervals.

[0184] The interval determination module 550 is used to determine a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval.

[0185] The exposure adjustment module 560 is used to perform exposure adjustment processing according to the target shutter information and the target gain information corresponding to the target exposure sub-interval.

[0186] In this exemplary exposure adjustment device based on motion judgment, by acquiring the speed information of each pixel in the current frame image, the current frame image can be divided into a high-speed motion area and a low-speed motion area according to the speed information. According to the first speed of the high-speed motion area and in combination with the second speed of the low-speed motion area, the current frame image can be filtered to obtain the filtered brightness. According to the filtered brightness and the preset target brightness, the target gain multiple that needs to be adjusted can be determined. Then, according to the first speed of the high-speed motion area combined with the exposure upper limit, the preset exposure interval is segmented to generate exposure sub-intervals. Then, according to the target gain multiple and the gain information of each exposure sub-interval, the target exposure sub-interval can be determined from each exposure sub-interval. In this way, the target shutter information and target gain information corresponding to the target exposure sub-interval can be used as target exposure parameters for exposure adjustment processing, so as to realize exposure adjustment based on motion judgment, avoid the interference of high-speed moving objects on the exposure adjustment process, and optimize the effect of automatic exposure adjustment.

[0187] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0188] The functions of each module may be found in the embodiment of the exposure adjustment method based on motion judgment, which will not be described in detail here.

[0189] See also Figure 6 , Figure 6 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102, and the processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above-mentioned exposure adjustment method embodiments based on motion judgment. In a specific implementation scenario, the electronic device 100 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 100 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0190] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-mentioned exposure adjustment method embodiments based on motion judgment. The processor 102 can also be called a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0191] In this exemplary electronic device, by acquiring the speed information of each pixel in the current frame image, the current frame image can be divided into a high-speed motion area and a low-speed motion area according to the speed information. According to the first speed of the high-speed motion area and in combination with the second speed of the low-speed motion area, the current frame image can be filtered to obtain the filtered brightness. According to the filtered brightness and the preset target brightness, the target gain multiple that needs to be adjusted can be determined. Then, according to the first speed of the high-speed motion area combined with the exposure upper limit, the preset exposure interval is segmented to generate exposure sub-intervals. Then, according to the target gain multiple and the gain information of each exposure sub-interval, the target exposure sub-interval can be determined from each exposure sub-interval. In this way, the target shutter information and target gain information corresponding to the target exposure sub-interval can be used as target exposure parameters for exposure adjustment processing, so as to realize exposure adjustment based on motion judgment, avoid interference of high-speed moving objects on the exposure adjustment process, and optimize the effect of automatic exposure adjustment.

[0192] See also Figure 7 , Figure 7 The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor, and the program instructions 111 are used to implement the steps in any of the above-mentioned exposure adjustment method embodiments based on motion judgment.

[0193] In this exemplary storage medium, by running the program instructions in the storage medium, the speed information of each pixel in the current frame image is obtained, and the current frame image can be divided into a high-speed motion area and a low-speed motion area according to the speed information. According to the first speed of the high-speed motion area, combined with the second speed of the low-speed motion area, the current frame image can be filtered to obtain the filtered brightness. According to the filtered brightness and the preset target brightness, the target gain multiple that needs to be adjusted can be determined. Then, according to the first speed of the high-speed motion area combined with the exposure upper limit, the preset exposure interval is segmented to generate exposure sub-intervals. Then, according to the target gain multiple and the gain information of each exposure sub-interval, the target exposure sub-interval can be determined from each exposure sub-interval. In this way, the target shutter information and target gain information corresponding to the target exposure sub-interval can be used as target exposure parameters for exposure adjustment processing, so as to realize exposure adjustment based on motion judgment, avoid interference of high-speed moving objects on the exposure adjustment process, and optimize the effect of automatic exposure adjustment.

[0194] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0195] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0196] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0197] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An exposure adjustment method based on motion judgment, characterized in that: The method comprises: Acquire speed information of each pixel in the current frame image, and determine a high-speed motion area and a low-speed motion area in the current frame image according to the speed information; Determining corresponding weight information according to the first speed of each pixel in the high-speed motion area and the second speed of each pixel in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image; Determining a gain multiple to be adjusted according to the filtering brightness and a preset target brightness, and determining a target gain multiple according to the gain multiple to be adjusted and a current gain multiple corresponding to the current frame image; Performing interval segmentation processing on the preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain a plurality of exposure sub-intervals; Determining a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval; Exposure adjustment processing is performed according to the target shutter information and the target gain information corresponding to the target exposure sub-interval.

2. The method according to claim 1, characterized in that The acquiring speed information of each pixel in the current frame image, and determining the high-speed motion area and the low-speed motion area in the current frame image according to the speed information, comprises: Obtaining the optical flow velocity of each pixel in the current frame image; Determine the speed segmentation threshold of the current frame image according to the optical flow speed of each pixel; The current frame image is segmented into the high-speed motion area and the low-speed motion area according to the speed segmentation threshold and the optical flow speed of each pixel.

3. The method according to claim 2, characterized in that The determining the speed segmentation threshold of the current frame image according to the optical flow speed of each pixel point includes: Calculate the average value of the optical flow velocity of each pixel to obtain the initial segmentation threshold of the current frame image; Segmenting the current frame image according to the initial segmentation threshold to obtain a first region and a second region in the current frame image; Calculating the average of the optical flow velocity of the first area and the average of the optical flow velocity of the second area to obtain a current segmentation threshold; In response to a difference between the initial segmentation threshold and the current segmentation threshold being smaller than a preset threshold difference, the current segmentation threshold is determined as the speed segmentation threshold.

4. The method according to claim 1, characterized in that The determining corresponding weight information according to the first speed of each pixel point in the high-speed motion area and the second speed of each pixel point in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image includes: Determine a first velocity mean value of the high-speed motion area according to the first velocity of each pixel point in the high-speed motion area, and determine a second velocity mean value of the low-speed motion area according to the second velocity of each pixel point in the low-speed motion area; Determine a brightness motion weight of each pixel according to the speed information of each pixel, the first speed average, the second speed average, and a preset motion weight; Determining the mean brightness of the current frame image according to the brightness motion weight and the brightness information of the current frame image; Determine a brightness filtering weight of the current frame image according to the first speed mean, a preset mean threshold and a preset filtering parameter; The mean brightness and the brightness filtering weight are input into a preset mean filter for filtering to obtain the filtered brightness.

5. The method according to claim 1, characterized in that The step of determining the gain multiple to be adjusted according to the filtering brightness and the preset target brightness, and determining the target gain multiple according to the gain multiple to be adjusted and the current gain multiple corresponding to the current frame image, comprises: Determining the gain multiple to be adjusted according to the ratio between the filtered brightness and the preset target brightness; Obtaining current shutter information and current gain information corresponding to the current frame image; Determine the current gain multiple according to the current shutter information and the current gain information; The target gain multiple is obtained by summing the gain multiple to be adjusted and the current gain multiple.

6. The method according to claim 1, characterized in that Before segmenting the preset exposure interval according to the exposure upper limit corresponding to the first speed to obtain a plurality of exposure sub-intervals, the method further includes: Determine a first velocity mean value of the high-speed motion area according to the first velocity of each pixel point in the high-speed motion area; Inputting the first speed mean into a preset mean filter for filtering to obtain a filtered speed; The exposure upper limit is determined according to the filtering speed and a preset speed parameter.

7. The method according to claim 1, characterized in that The determining a target exposure sub-interval from each exposure sub-interval according to the target gain multiple and the gain information corresponding to each exposure sub-interval includes: Determine the segmented gain multiple corresponding to each exposure sub-interval according to the shutter information and gain information corresponding to each exposure sub-interval; The target exposure sub-interval is determined according to the target gain multiple and the segmented gain multiple.

8. The method according to claim 1, characterized in that After determining corresponding weight information according to the first speed of each pixel in the high-speed motion area and the second speed of each pixel in the low-speed motion area, and filtering the brightness information of the current frame image according to the weight information to obtain the filtered brightness of the current frame image, the method further includes: Determining a brightness multiple of the current frame image according to shutter information and gain information of the current frame image; Determine the ambient brightness of the current frame image according to the filtered brightness and the brightness multiple; The working mode of the image acquisition device used to acquire the current frame image is controlled according to the ambient brightness and a preset brightness threshold; the working mode includes a first mode and a second mode.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images