An automatic exposure method for binocular cameras

Abstract

This application discloses an automatic exposure method for a stereo camera. Based on the disparity quality evaluation index of the images acquired by the stereo camera, the method adjusts the automatic exposure target brightness value of the stereo camera. This method is based on a steady-state optimal brightness target value tracking mechanism, using the brightness value of the image corresponding to the maximum value of the disparity quality evaluation index as the new automatic exposure target brightness value, and then applying it to the subsequent automatic exposure adjustment process of the stereo camera. This application optimizes the image acquisition effect of the stereo camera, enabling the automatic exposure algorithm to automatically track scene changes. It effectively overcomes the problem in existing technologies where manually setting the automatic exposure brightness target value cannot meet the requirements of full scene coverage and scene changes during movement.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an automatic exposure method for a binocular camera. Background Technology

[0002] Binocular stereo vision technology is widely used in autonomous driving, such as lane detection and obstacle detection. When using binocular stereo vision for target detection, the disparity quality evaluation index in the image is a key indicator for judging the quality of image acquisition. This index can be measured by factors such as disparity density and the continuity of object edges in the disparity map. A higher disparity quality evaluation index value indicates more information available for target detection and higher image quality. During image acquisition, exposure time and exposure gain directly affect the disparity quality evaluation index. Especially during the movement of a mobile platform equipped with binocular cameras, there may be scenarios with inconsistent lighting intensity, such as a vehicle entering / exiting an underground parking garage or a mobile robot moving from outdoors to indoors. In such cases, automatic exposure processing of the binocular camera is necessary.

[0003] Automatic exposure (AE) technology typically uses a PID (proportional-integral-derivative) algorithm to calculate the exposure time and gain for the next frame based on the target brightness value and the current brightness value of the image (or a region within the image). When the difference between the target brightness value and the current frame brightness value is less than a preset threshold, the exposure is considered to have reached a steady state and no further adjustments are made. This continues until the difference between the brightness of a particular frame and the target brightness exceeds the threshold, at which point PID adjustment is restarted.

[0004] For example, patent CN113382143A discloses an automatic exposure adjustment method for a binocular camera on a fire-fighting robot. This method involves gradually increasing the exposure time from small to large while maintaining a fixed exposure gain, and then gradually increasing the exposure gain again to capture multi-sample image data of the binocular camera in complex scenes under extreme conditions. Considering the special structure of the binocular camera and the information overlap of the binocular imaging system, effective mutual information entropy analysis is performed on the multi-sample image data. Disparity map quality analysis is conducted on the stereo matching of the binocular images using a disparity map quality analysis factor to determine the image brightness target for automatic exposure adjustment of the fire-fighting robot's binocular vision system. Then, by comparing the calculated brightness information of the current frame with the target brightness information, a variable step size exposure adjustment time is obtained to automatically control the camera's exposure, thereby enabling the binocular camera to quickly adapt to different environments.

[0005] In existing technologies, the common approach for automatic exposure processing with binocular cameras is to set a fixed target brightness value and threshold within the camera. When the scene changes, the exposure time and exposure gain are adjusted using methods such as PID control to ensure that the brightness value of the image captured by the binocular camera in the current scene remains within the set target brightness value and threshold range. However, the ambient brightness of the binocular camera is often unknown. This means that the set target brightness value may not be suitable for the current environment, and it cannot guarantee that the parallax quality evaluation index in the captured image is optimal, i.e., it cannot guarantee that the image captured by the binocular camera contains the most target detection information.

[0006] In addition, manually set exposure parameters cannot meet the needs of full scene coverage and the tracking of scene changes during movement, such as switching between bright and dark scenes. Summary of the Invention

[0007] The technical problem to be solved by this application is: how to adjust the target brightness value during the automatic exposure process of a binocular camera in order to ensure the parallax quality evaluation index of the images acquired by the binocular camera.

[0008] This application introduces a secondary tracking mechanism for target brightness values ​​under steady-state automatic exposure of a binocular camera, and proposes an automatic exposure method for a binocular camera that can optimize the image acquisition effect and enable the automatic exposure algorithm to automatically track scene changes. The method includes: adjusting the target brightness value of the automatic exposure of the binocular camera according to the parallax quality evaluation index of the image acquired by the binocular camera.

[0009] In some embodiments, adjusting the target brightness value of the automatic exposure of the binocular camera based on the parallax quality evaluation index of the image acquired by the binocular camera includes:

[0010] Step 1: Acquire the current frame image captured by the binocular camera in the current scene, record the imaging brightness value corresponding to the current frame image, and calculate the disparity quality evaluation index of the current frame image.

[0011] Step 2: When the binocular camera is determined to have entered the steady state of exposure, compare the parallax quality evaluation index of N consecutive frames, and update the automatic exposure target brightness value of the binocular camera according to the imaging brightness value of the image corresponding to the maximum value of the parallax quality evaluation index in the N consecutive frames.

[0012] Here, N consecutive frames are the N frames preceding the current frame.

[0013] In some embodiments, the method for determining in step 2 that the binocular camera has entered an exposure steady state includes:

[0014] Step 2.1: When the absolute value of the difference between the imaging brightness value of the current frame image and the automatic exposure target brightness value of the stereo camera is less than or equal to the set first threshold, count the number of image frames acquired by the stereo camera and record the exposure time and exposure gain corresponding to each frame image.

[0015] Step 2.2: When the count value of the number of image frames is greater than or equal to M, determine whether the change in the value of the exposure duration and / or exposure gain is less than or equal to the second threshold. If so, determine that the stereo camera has entered the exposure steady state.

[0016] If not, repeat step 1.

[0017] In some embodiments, in step 2, the disparity quality evaluation index of N consecutive frames is compared, and the imaging brightness value of the binocular camera's automatic exposure target brightness value is updated based on the imaging brightness value of the image corresponding to the maximum value of the disparity quality evaluation index in the N consecutive frames. Specifically, this includes:

[0018] Step 2.3: Compare the disparity quality evaluation indexes of N consecutive frames of images, and select the maximum value among the disparity quality evaluation indexes corresponding to the N consecutive frames of images, and record it as the maximum value of the disparity quality evaluation index.

[0019] Step 2.4: Determine whether the disparity quality evaluation index of the current frame image is less than the maximum value of the disparity quality evaluation index;

[0020] If so, update the brightness value of the image corresponding to the maximum value of the parallax quality evaluation index to the automatic exposure target brightness value;

[0021] Otherwise, repeat step 1 to continue.

[0022] In some embodiments, the method further includes:

[0023] Set the exposure parameters for the binocular camera, and set the initial automatic exposure target brightness value and the corresponding first threshold;

[0024] Based on the current frame image, adjust the exposure time and exposure gain of the stereo camera to adjust the image brightness value of the stereo camera to the target brightness range;

[0025] The target brightness range is determined by the initial automatic exposure target brightness value and the first threshold.

[0026] In some embodiments, the method further includes:

[0027] Calculate the weighted brightness value of the current frame image and record the weighted brightness value as the imaging brightness value;

[0028] Calculate the absolute value of the difference between the image brightness value and the initial automatic exposure target brightness value;

[0029] Determine whether the absolute value of the difference is greater than a set first threshold;

[0030] When the absolute value of the difference is greater than the first threshold, the exposure duration and exposure gain of the stereo camera are adjusted according to the current frame image to adjust the imaging brightness value of the stereo camera to the target brightness range.

[0031] When the absolute value of the difference is less than or equal to the first threshold, it is determined whether the stereo camera has entered the steady state of exposure.

[0032] In some embodiments, calculating the weighted brightness value of the current frame image includes:

[0033] Step 1.1: Divide the current frame image into R*C grids, calculate the average brightness of all pixels in each grid, and generate an R*C dimensional average brightness matrix according to the grid order and the corresponding calculated average brightness.

[0034] Step 1.2: Combine the R*C dimensional average brightness matrix into n regions, labeled as regions Z1 to Zn respectively, and then average the average brightness of the grid in each region to determine the brightness of each region, which are denoted as brightness B1 to Bn respectively.

[0035] Step 1.3: Randomly assign different weights to each region in the range Z1 to Zn, denoted as weights W1 to Wn. Multiply the brightness values ​​B1 to Bn by their corresponding weights W1 to Wn to calculate the weighted brightness values. The corresponding calculation formula is as follows:

[0036] The weighted brightness value of the image = W1*Bn + W2*B2 + ... + Wn*Bn.

[0037] In some embodiments, before averaging the average brightness of the grid in each region Z1 to Zn in step 1.2, the method further includes:

[0038] Remove overexposed and underexposed grids from regions Z1 to Zn;

[0039] Among them, the overexposed grid is the grid in region Z1 to Zn where the image brightness value is greater than the preset brightness value A;

[0040] The underexposed grid is the grid in region Z1 to Zn where the image brightness value is less than the preset brightness value B;

[0041] The preset brightness value A is greater than the preset brightness value B.

[0042] In some embodiments, M is a positive integer greater than 3 and less than 10.

[0043] In some embodiments, N is a positive integer greater than 10 and less than 50.

[0044] The beneficial effects of this application are:

[0045] This application introduces a tracking mechanism for the optimal target brightness value under steady-state conditions. When the stereo camera enters a steady-state exposure, it compares the current frame with the previous N consecutive frames to determine whether the maximum value of the disparity quality evaluation index in the previous N consecutive frames is higher than the disparity quality evaluation index of the current frame. If the maximum value of the disparity quality evaluation index in the most recent N frames is higher than the disparity quality evaluation index of the current frame, then the brightness value of the image corresponding to the maximum value of the disparity quality evaluation index is used as the new automatic exposure target brightness value (Target) for the stereo camera and used in the subsequent automatic exposure adjustment process of the stereo camera. Through this method provided by this application, the image acquisition effect of the stereo camera is optimized, and the automatic exposure algorithm can automatically track scene changes.

[0046] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0047] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following detailed description of the embodiments in conjunction with the accompanying drawings is provided, but does not constitute any limitation on this application.

[0049] Figure 1 This is a flowchart illustrating the binocular camera automatic exposure method of this application;

[0050] Figure 2 This is a schematic diagram of the image used to calculate the weighted brightness value in this application;

[0051] Figure 3 This is a flowchart illustrating the automatic exposure method for a binocular camera in a preferred embodiment;

[0052] Figure 4 This is a schematic diagram illustrating the parallax effect before introducing target tracking in an outdoor scene.

[0053] Figure 5 This is a schematic diagram illustrating the parallax effect after introducing Target tracking in an outdoor scene (currently). Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0055] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0056] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0057] Example 1

[0058] In current mainstream automatic exposure methods, a fixed target brightness value and threshold are usually set in the stereo camera. The difference between the brightness value of the current frame captured by the stereo camera and the set target brightness value is determined. If the difference is less than the set threshold, the stereo camera enters a steady-state exposure. Otherwise, a PID control method is used to adjust the exposure duration and exposure gain of the next frame image obtained by the stereo camera, so that the image brightness value captured by the stereo camera in the current scene is within the set target brightness value and threshold range.

[0059] However, the above methods lack the ability to judge scene changes (such as changes in brightness). Considering the uncertainty of ambient brightness changes in the current scene, if only a fixed target brightness value is set during the automatic exposure process, the imaging effect of the binocular camera is not ideal. In particular, for the target recognition process, the parallax information contained in the image for target detection is often not optimal. Therefore, when the scene changes, the existing automatic exposure method may lead to the loss of target detection information.

[0060] Therefore, this embodiment proposes a method for adjusting the automatic exposure target brightness value of a binocular camera, ensuring the parallax quality evaluation index of the images acquired by the binocular camera during scene judgment and tracking. The method includes:

[0061] Based on the disparity quality evaluation index of the images acquired by the binocular camera, adjust the target brightness value of the automatic exposure of the binocular camera. The disparity quality evaluation index can be the disparity density and / or the continuity of object edges in the disparity map, or other indicators that can evaluate the quality of the disparity map.

[0062] It should be noted that in this embodiment, a brute-force search method can be used to adjust the target brightness value of the automatic exposure of the binocular camera, so as to make the disparity quality evaluation index of the image acquired by the binocular camera better, and ensure that the disparity map obtained in the scene judgment and tracking process has more useful information for target recognition.

[0063] Combined with appendix Figure 1 The above-mentioned automatic exposure method for binocular cameras includes the following steps:

[0064] Step 1: Acquire the current frame image captured by the binocular camera in the current scene, record the imaging brightness value corresponding to the current frame image, and calculate the disparity quality evaluation index of the current frame image.

[0065] Step 2: When the binocular camera is determined to have entered the steady state of exposure, compare the parallax quality evaluation index of N consecutive frames, and update the automatic exposure target brightness value of the binocular camera according to the imaging brightness value of the image corresponding to the maximum value of the parallax quality evaluation index in the N consecutive frames.

[0066] Among them, the consecutive N frames are the N frames preceding the current frame image captured by the stereo camera.

[0067] This embodiment introduces an optimal target brightness value tracking mechanism, using the parallax quality evaluation index as the basis for updating the automatic exposure target brightness value of the stereo camera. When the stereo camera enters a steady-state exposure, the parallax quality evaluation index of N consecutive frames is compared. Based on the imaging brightness value of the image corresponding to the maximum value of the parallax quality evaluation index in the N consecutive frames, the automatic exposure target brightness value of the stereo camera is updated and used in the subsequent automatic exposure adjustment process of the stereo camera. This optimizes the image acquisition effect of the stereo camera, enabling not only automatic tracking of scene changes by the automatic exposure algorithm, but also ensuring the parallax quality evaluation index of the images acquired by the stereo camera, thus solving the problem of image target detection information loss caused by changes in scene brightness.

[0068] In some embodiments, to avoid frequent adjustments to the target brightness value, if the imaging brightness value of the current frame image meets the limitation of the target brightness value and the first threshold, it is also necessary to determine the stability of the M consecutive frames to determine whether the stereo camera has entered an exposure steady state. The method for determining whether the stereo camera has entered an exposure steady state includes:

[0069] Step 2.1: When the absolute value of the difference between the imaging brightness value of the current frame image and the automatic exposure target brightness value of the stereo camera is less than or equal to the set first threshold, count the number of image frames acquired by the stereo camera and record the exposure time and exposure gain corresponding to each frame image; when the absolute value of the above difference is greater than the first threshold, it is determined that the stereo camera has not entered the exposure steady state, and step 1 is executed.

[0070] Step 2.2: When the count value of the number of image frames is greater than or equal to M, determine whether the change in the exposure duration and / or exposure gain of the consecutive M frames is less than or equal to the second threshold. If so, determine that the stereo camera has entered the exposure steady state.

[0071] If not, repeat step 1. When the count of the number of image frames is less than M, it is considered that the current scene where the stereo camera is located has changed, causing the stereo camera to be unable to enter the exposure steady state, and step 1 above needs to be repeated.

[0072] It should be noted that the second threshold can be 0.

[0073] In some embodiments, the imaging brightness value of the image with the largest disparity quality evaluation index among the previous N consecutive frames can be directly selected as the new automatic exposure target brightness value for the stereo camera. However, to ensure the accuracy of the selected target brightness value, the maximum value of the disparity quality evaluation index among the previous N consecutive frames can be compared with the disparity quality evaluation index of the current frame image to update the automatic exposure target brightness value of the stereo camera. Specifically, this includes:

[0074] Step 2.3: Compare the disparity quality evaluation indexes of N consecutive frames of images, and select the maximum value among the disparity quality evaluation indexes corresponding to the N consecutive frames of images, and record it as the maximum value of the disparity quality evaluation index.

[0075] Step 2.4: Determine whether the disparity quality evaluation index of the current frame image is less than the maximum value of the disparity quality evaluation index;

[0076] If so, update the brightness value of the image corresponding to the maximum value of the parallax quality evaluation index to the automatic exposure target brightness value;

[0077] Otherwise, repeat step 1 to continue.

[0078] In this embodiment, by introducing a secondary tracking mechanism for target brightness value under automatic exposure steady state, the problem of existing binocular camera automatic exposure technology being unable to automatically track target brightness value when the environment changes due to the inability to set a fixed target brightness value is solved. This results in the inability to guarantee the best parallax quality evaluation index in the image acquired by the binocular camera, and the possibility of image detection information loss.

[0079] Example 2

[0080] Based on Example 1, this example specifically introduces an automatic exposure method for a binocular camera, and uses parallax density as a parallax quality evaluation index for explanation, in conjunction with the appendix. Figure 3 As shown, the automatic exposure method for this binocular camera is executed as follows:

[0081] Before acquiring images using a stereo camera, the exposure parameters of the stereo camera are set. An initial automatic exposure target brightness value and a corresponding first threshold are set. Based on the current frame image, the exposure time and exposure gain of the stereo camera can be adjusted using PID to adjust the imaging brightness value of the stereo camera to the target brightness range. The target brightness range is determined by the initial automatic exposure target brightness value and the first threshold.

[0082] First, acquire the current frame image captured by the binocular camera in the current scene, record the imaging brightness value corresponding to the current frame image, and calculate the disparity density of the current frame image.

[0083] For example, in natural scenes, to ensure the clarity of images captured by the stereo camera, the initial automatic exposure target brightness value is set to 2000, with a corresponding first threshold of 400. After the stereo camera is powered on, a PID control method is used to adjust the exposure time and exposure gain, adjusting the brightness value of the image to a range of 2000±400 to ensure image clarity. Furthermore, if the scene remains unchanged, the exposure time and exposure gain will remain constant until the scene changes.

[0084] In this embodiment, taking the example of a mobile platform equipped with a binocular camera moving from indoors to outdoors, as the mobile platform gradually moves from indoors to outdoors, the ambient light intensity also changes, and the image captured by the binocular camera will be overexposed. At this time, the exposure time and exposure gain of the image can be reduced by using PID adjustment, and the brightness of the image captured by the binocular camera in the current scene can be readjusted to the range of 2000±400.

[0085] Of course, in some cases, the preset initial auto exposure target brightness value and the brightness range covered by the first threshold may not achieve the optimal parallax density. For example, when the initial auto exposure target brightness value and the first threshold are set to 2000 and 400 respectively, the range (1600, 2400) covered by this setting may not achieve the highest parallax density. In this case, the brightness value corresponding to the optimal parallax density may be 800.

[0086] Referring to Table 1 below, by observing the data in Table 1, it can be found that when the initial target brightness value (Target) is set to 2000, the stereo camera automatically adjusts the exposure after starting. The exposure duration and gain value remain unchanged from frame 9 to frame 13, thus completing the automatic exposure of the stereo camera. The stereo camera can work normally and acquire image data.

[0087] However, by comparing the disparity density of the previous frames, it was found that the disparity density was the largest in the 7th frame, with a disparity density of 74%. At this time, the image contained the most information that could be used for target detection / recognition. Therefore, based on the purpose of target detection / recognition, if we want the image acquired by the stereo camera to have the largest disparity density, we need to adjust the initial target, i.e., the new target brightness, in order to optimize the disparity density of the image acquired by the stereo camera.

[0088] Table 1 Steady-State Target Tracking Cases

[0089] Frame number Target Brightness of this frame Exposure duration Exposure gain Parallax density (%) 1 2000 0000 0354 0100 00 2 2000 0261 0474 0100 23 3 2000 0322 0613 0100 42 4 2000 0379 0758 0100 52 5 2000 0442 1072 0100 58 6 2000 0513 1390 0100 72 7 2000 0762 1680 0125 74 8 2000 0810 1850 0149 71 9 2000 0943 1850 0188 73 10 2000 1065 1850 0188 72 11 2000 1120 1850 0188 72 12 2000 1119 1850 0188 72 13 2000 1118 1850 0188 72

[0090] As the mobile platform moves, the ambient light changes accordingly. In order to ensure the exposure effect of the images captured by the binocular camera, the exposure time and exposure gain need to be adjusted. Therefore, in some embodiments, a weighted brightness value is introduced as the basis for adjustment.

[0091] Specifically, the weighted brightness value of the current frame image acquired by the binocular camera is calculated and recorded as the imaging brightness value.

[0092] Combined with appendix Figure 2 As shown, Figure 2 This is a schematic diagram of an image used in this application to calculate the weighted brightness value. In this embodiment, a single frame of a 1280*720 pixel image is used as an example for calculation.

[0093] Step 1.1: Divide the image into R*C grids, as follows: Figure 2 The system is divided into 8*8 grids. The average brightness of all pixels in each grid (0-255 / equivalent to grayscale) is calculated. Based on the grid order and the corresponding calculated average brightness, an 8*8 dimensional average brightness matrix is ​​generated.

[0094] Step 1.2: Combine the 8*8 dimensional average brightness matrix into n regions, such as... Figure 2 The grid is divided into 5 regions, labeled Z1 to Z5 respectively. The average brightness of the grid in each region is then averaged to determine the brightness of each region, which is then labeled B1 to B5.

[0095] Step 1.3: Randomly assign different weights to regions Z1 to Z5, and denot them as weights W1 to W5 (e.g., W1 = 0.05, W2 = 0.1, W3 = 0.6, W4 = 0.1, W5 = 0.15, ensuring that W1 + W2 + W3 + W4 + W5 = 1.0). Then, multiply the brightness values ​​B1 to B5 by their corresponding weights W1 to W5 to obtain the weighted brightness values ​​of that frame. The calculation formula is as follows:

[0096] The weighted brightness value of the image = W1*B1 + W2*B2 + W3*B3 + W4*B4 + W5*B5.

[0097] Before averaging the average brightness of the grids in each region Z1 to Z5, it is necessary to remove overexposed and underexposed grids in regions Z1 to Z5.

[0098] It should be noted that overexposed and underexposed grids can be understood as follows: an overexposed grid is a grid in region Z1 to Zn where the image brightness value is greater than a preset brightness value A; an underexposed grid is a grid in region Z1 to Zn where the image brightness value is less than a preset brightness value B; the preset brightness value A is greater than the preset brightness value B. For example, A can be set to 235 and B to 20.

[0099] Therefore, in this embodiment, in each grid corresponding to regions Z1 to Z5, grids with a brightness greater than 235 are overexposed grids, and grids with a brightness less than 20 are underexposed grids. For example, in... Figure 2 In the 16 grids of Z1, after removing one overexposed grid, 15 normal grids remain. The average brightness of these 15 normal grids is the brightness of Z1.

[0100] In addition, in this embodiment, the brightness values ​​in Table 1 are obtained by multiplying the brightness range of 0-255 by 21 (multiplying by 21). However, this is only one way of calculating brightness. This application does not limit the specific method of brightness weighting and statistics, and the grayscale values ​​of 0-255 can also be directly used as the adjustment parameters of the exposure algorithm.

[0101] Furthermore, the absolute value of the difference between the obtained imaging brightness value and the initial automatic exposure target brightness value is calculated;

[0102] Determine whether the absolute value of the difference is greater than a set first threshold;

[0103] When the absolute value of the difference is greater than the first threshold, the exposure duration and exposure gain of the stereo camera are adjusted according to the current frame image to adjust the imaging brightness value of the stereo camera to the target brightness range.

[0104] When the absolute value of the difference is less than or equal to the first threshold, it is determined whether the stereo camera has entered the steady state of exposure. The specific determination method will not be elaborated here.

[0105] The PID algorithm for adjusting the exposure duration and exposure gain of the current frame image includes the following steps:

[0106] Step 101: First, obtain the target brightness and the current frame image brightness;

[0107] Step 102: Set P (proportional coefficient), I (integral coefficient), D (derivative coefficient), initial exposure time / gain, and PID start threshold;

[0108] Step 103: Set the previous frame error_last = 0 for the differential term and the error history error_history = 0 for the integral term;

[0109] Step 104: Error = |Target Brightness - Current Frame Image Brightness|;

[0110] Step 105: Update the moving average error: error_ma = cumulative coefficient accu_factor * moving average error error_ma + (1 - cumulative coefficient accu_factor) * error;

[0111] Step 106: Determine whether the moving average error update error_ma is greater than the PID start threshold. If yes, proceed to step 107; otherwise, stop executing the PID algorithm adjustment process and obtain the exposure time Expo and exposure gain Gain.

[0112] Step 107: Error history error_history = error_history + error_diff = error - previous frame error error_last;

[0113] Step 108: Calculate the exposure adjustment amounts corresponding to P / I / D respectively;

[0114] Step 109: Determine if the exposure time is less than the upper limit. If so, continue to step 110. If not, adjust the exposure gain and obtain the exposure time Expo and the adjusted exposure gain Gain.

[0115] Step 110: Adjust the exposure time and obtain the adjusted exposure time Expo and exposure gain Gain.

[0116] Secondly, when the stereo camera is determined to have entered a steady state of exposure, the disparity density of N consecutive frames is compared, and the imaging brightness value of the stereo camera is updated according to the image brightness value of the image corresponding to the maximum disparity density in the N consecutive frames.

[0117] Specifically, methods for determining whether a stereo camera has entered a steady-state exposure include:

[0118] When the absolute value of the difference between the imaging brightness value of the current frame image and the automatic exposure target brightness value of the stereo camera is less than or equal to the set first threshold, the number of image frames acquired by the stereo camera is counted, and the exposure time and exposure gain corresponding to each frame image are recorded.

[0119] When the count value of the number of image frames is greater than or equal to M, determine whether the change in the value of exposure duration and / or exposure gain is less than or equal to the second threshold. If so, determine that the stereo camera has entered the exposure steady state.

[0120] If not, then acquire the next frame and continue execution.

[0121] Where M can be a positive integer greater than 3 and less than 10. For example, in this embodiment, the exposure duration and exposure gain values ​​corresponding to the five consecutive frames from frame 9 to frame 13 are selected to determine whether there are any changes. As shown in Table 1, the exposure duration of the five consecutive frames from frame 9 to frame 13 is 1850 and the exposure gain is 188, which indicates that the stereo camera has entered the automatic exposure steady state.

[0122] In some embodiments, the automatic exposure target brightness value of the binocular camera is updated by comparing the disparity density of N consecutive frames and based on the imaging brightness value of the image corresponding to the maximum disparity density in the N consecutive frames. This is specifically performed as follows:

[0123] Compare the disparity density of N consecutive frames and select the maximum value among the disparity density of the N consecutive frames, which is denoted as the maximum disparity density.

[0124] Determine whether the disparity density of the current frame image is less than the maximum disparity density.

[0125] If so, update the brightness value of the image corresponding to the maximum parallax density to the automatic exposure target brightness value;

[0126] Otherwise, the automatic exposure target brightness value will not be updated, and the next frame image will be acquired again.

[0127] Wherein, N can be a positive integer greater than 10 and less than 50. In this embodiment, N is preferably 32 frames, that is, the 32 consecutive frames before the current frame are selected as the optimal range of disparity density values, and the optimal disparity density within the previous 32 frames is calculated for each frame, and the corresponding brightness value is recorded. This is updated cyclically as a candidate value for the next adjustment of the target brightness value.

[0128] In this embodiment, when the automatic exposure reaches a steady state, it is determined whether the maximum disparity density of the most recent 32 frames of the current frame image is higher than the disparity density of the current frame image. If the maximum disparity density of the most recent 32 frames is higher than the disparity density of the current frame image, the brightness value of the image corresponding to the maximum disparity density is taken as the new target brightness value, and the new target brightness value is used for the automatic exposure adjustment process of subsequent frames.

[0129] By repeatedly performing the above steps, the automatic exposure algorithm can automatically track scene changes (environmental changes), which facilitates the adjustment of target brightness values ​​by the binocular camera during automatic exposure. This ensures that the disparity density in the images acquired by the binocular camera is optimal, thereby increasing the amount of target detection information in the images. Furthermore, the above method can effectively overcome the limitations of manual target brightness setting methods in existing technologies, which cannot meet the requirements of full scene coverage and tracking of scene changes during motion.

[0130] This embodiment also compares with existing automatic exposure solutions. Specifically, during the movement of the mobile platform equipped with a stereo camera, when the scene within the stereo camera's field of view changes, such as overexposure when the stereo camera is facing a light source, the existing automatic exposure solutions are compared with the technical solution provided in this embodiment (i.e., the automatic exposure solution incorporating target tracking), as shown in the attached figure. Figure 4 and Figure 5 As shown.

[0131] The test results show that, in an outdoor scene, the disparity density of the same frame image is 13% with target tracking and 57% with target tracking. The automatic exposure method using the introduced steady-state target tracking mechanism significantly improves the disparity density compared to existing automatic exposure methods without target tracking, thus helping to enhance the accuracy of target detection by binocular cameras when the scene changes.

[0132] By introducing a tracking mechanism for the optimal target brightness value under steady state, the binocular camera can automatically track the optimal target brightness value of the most recent frame image, creatively solving the scene tracking requirements of autonomous driving scenarios (such as robots and autonomous vehicles).

[0133] The above-described embodiments are preferred embodiments of this application and are only used to facilitate the illustration of this application. They are not intended to limit this application in any way. Any person with ordinary knowledge in the art can make equivalent embodiments by making partial modifications or alterations to the technical content disclosed in this application without departing from the scope of the technical features of this application. Such equivalent embodiments are still within the scope of the technical features of this application.

Claims

1. An automatic exposure method for a binocular camera, characterized in that, The method includes: Based on the parallax quality evaluation index of the images acquired by the binocular camera, adjust the automatic exposure target brightness value of the binocular camera; the parallax quality evaluation index is the parallax density and / or the continuity of object edges in the parallax map; The step of adjusting the automatic exposure target brightness value of the binocular camera based on the parallax quality evaluation index of the image acquired by the binocular camera includes: Step 1: Acquire the current frame image captured by the binocular camera in the current scene, record the imaging brightness value corresponding to the current frame image, and calculate the disparity quality evaluation index of the current frame image; Step 2: When the binocular camera is determined to have entered the exposure steady state, the disparity quality evaluation index of N consecutive frames is compared, and the automatic exposure target brightness value of the binocular camera is updated according to the imaging brightness value of the image corresponding to the maximum value of the disparity quality evaluation index in the N consecutive frames. Wherein, the consecutive N frames are the N frames preceding the current frame; In step 2, the method for determining that the stereo camera has entered a steady-state exposure includes: Step 2.1: When the absolute value of the difference between the imaging brightness value of the current frame image and the automatic exposure target brightness value of the binocular camera is less than or equal to the set first threshold, the number of image frames acquired by the binocular camera is counted, and the exposure duration and exposure gain corresponding to each frame image are recorded. Step 2.2: When the count value of the number of image frames is greater than or equal to M, determine whether the change in the exposure duration and / or the exposure gain is less than or equal to the second threshold. If yes, determine that the binocular camera has entered the exposure steady state; if no, repeat step 1.

2. The automatic exposure method for a binocular camera according to claim 1, characterized in that, In step 2, the disparity quality evaluation index of N consecutive frames is compared, and the automatic exposure target brightness value of the binocular camera is updated according to the imaging brightness value of the image corresponding to the maximum value of the disparity quality evaluation index in the N consecutive frames. Specifically, this includes: Step 2.3: Compare the disparity quality evaluation indexes of the consecutive N frames of images, and select the maximum value among the disparity quality evaluation indexes corresponding to the consecutive N frames of images, and record it as the maximum value of the disparity quality evaluation index. Step 2.4: Determine whether the disparity quality evaluation index of the current frame image is less than the maximum value of the disparity quality evaluation index; If so, the brightness value of the image corresponding to the maximum value of the parallax quality evaluation index is updated to the brightness value of the automatic exposure target; Otherwise, repeat step 1 to continue.

3. The automatic exposure method for a binocular camera according to claim 1 or 2, characterized in that, The method further includes: setting the exposure parameters of the binocular camera, and setting an initial automatic exposure target brightness value and a corresponding first threshold; Based on the current frame image, adjust the exposure duration and exposure gain of the binocular camera to adjust the imaging brightness value of the binocular camera to the target brightness range; The target brightness range is determined by the initial automatic exposure target brightness value and the first threshold.

4. The automatic exposure method for a binocular camera according to claim 3, characterized in that, The method further includes: Calculate the weighted brightness value of the current frame image, and record the weighted brightness value as the imaging brightness value; Calculate the absolute value of the difference between the image brightness value and the initial automatic exposure target brightness value; Determine whether the absolute value of the difference is greater than the set first threshold; when the absolute value of the difference is greater than the first threshold, adjust the exposure duration and exposure gain of the stereo camera according to the current frame image, and adjust the imaging brightness value of the stereo camera to the target brightness range; When the absolute value of the difference is less than or equal to the first threshold, it is determined whether the binocular camera has entered a steady-state exposure state.

5. The automatic exposure method for a binocular camera according to claim 4, characterized in that, The calculation of the weighted brightness value of the current frame image includes: Step 1.1: Divide the current frame image into R*C grids, calculate the average brightness of all pixels in each grid, and generate an R*C dimensional average brightness matrix according to the grid order and the corresponding calculated average brightness. Step 1.2: Combine the R*C dimensional average brightness matrix into n regions, which are labeled as regions Z1 to Zn respectively. Then, average the average brightness of the grid in each region to determine the brightness of each region, which are denoted as brightness B1 to Bn respectively. Step 1.3: Randomly assign different weights to each region Z1 to Zn, denoted as weights W1 to Wn. Multiply the brightness values ​​B1 to Bn by their corresponding weights W1 to Wn to calculate the weighted brightness value. The corresponding calculation formula is as follows: The weighted brightness value of the image = W1*Bn + W2*B2 + ... + Wn*Bn.

6. The automatic exposure method for a binocular camera according to claim 5, characterized in that, In step 1.2, before averaging the average brightness of the grid in each region Z1 to Zn, the method further includes: Remove overexposed and underexposed grids from the region Z1 to Zn; Wherein, the overexposed grid is the grid in the region Z1 to Zn where the image brightness value is greater than a preset brightness value A; The underexposed grid is the grid in the region Z1 to Zn where the image brightness value is less than a preset brightness value B; The value of the preset brightness value A is greater than the value of the preset brightness value B.

7. The automatic exposure method for a binocular camera according to claim 1, characterized in that, M is a positive integer greater than 3 and less than 10.

8. The automatic exposure method for a binocular camera according to claim 1, characterized in that, N is a positive integer greater than 10 and less than 50.

Images