An image exposure gain compensation system and method for binocular stitched images

Abstract

The application provides a system and method for image exposure gain compensation of binocular spliced images, the system comprising a data module, a coordinate mapping module, a brightness statistics module, an image gain compensation coefficient calculation module, a pixel gain coefficient calculation module and an image gain compensation output module. The method comprises the following steps: S1, obtaining a coordinate mapping file of a binocular spliced image to a binocular camera source image; S2, obtaining a pixel value of the binocular spliced image according to the coordinate mapping file; S3, obtaining a brightness mean value luma_mean of an overlapping area of the binocular spliced image corresponding to the binocular camera source image; S4, obtaining a gain compensation coefficient gainCorrectionRation corresponding to left and right source images of the binocular camera; S5, calculating a gain coefficient gainRatio of a current pixel value of a gain compensation area of the binocular spliced image; and S6, image gain compensation output value=value*gainRation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image exposure gain compensation system and method for binocular stitched images. Background Technology

[0002] In existing technologies, commonly used image processing techniques include binocular image stitching and automatic exposure. Binocular image stitching involves stitching together images from two cameras with overlapping areas to create a composite image. Exposure gain compensation is necessary because the exposure values ​​of the two cameras may differ, resulting in noticeable brightness differences between the two images along the stitching seam, which negatively impacts the user's viewing experience.

[0003] Considering that some in-vehicle platforms, security cameras, and mobile robots often require images with a large field of view to observe the surrounding environment, and given the relatively small field of view of a single lens, binocular stitching technology is needed to overcome the limitations of the field of view of a single image sensor and obtain images with a wider field of view. However, because the subject scene and related exposure configuration of the two cameras may differ, resulting in different exposure values ​​for the binocular cameras, the two images from the binocular cameras may show a significant brightness difference along the stitching seam, which is detrimental to the user's viewing experience. Therefore, it is necessary to perform gain compensation for the exposure of the stitched image.

[0004] In other words, the drawbacks of existing technologies are:

[0005] 1. Exposure gain compensation was not performed on the images stitched together from the binocular cameras.

[0006] 2. The exposure gain for image stitching from two cameras did not take into account the exposure conditions of both cameras.

[0007] 3. The cases of horizontal and vertical image stitching were not comprehensively considered. Summary of the Invention

[0008] To address the aforementioned problems, the present invention aims to provide an automatic exposure gain compensation system and method for binocular stitched images, specifically addressing the issue of varying exposure gains. This includes:

[0009] 1. Implement exposure gain compensation for images stitched together by binocular cameras to optimize the user's viewing experience of the stitched images.

[0010] 2. Gain compensation is performed on the stitched image from both cameras, taking into account the exposure conditions of both cameras.

[0011] 3. Exposure gain compensation is performed by comprehensively considering both horizontal and vertical image stitching.

[0012] 4. Gradual linear compensation of exposure gain allows for a smoother transition of exposure gain in binocular stitched images.

[0013] Specifically, this method provides an image exposure gain compensation system for binocular stitched images. The system includes: a data module, a coordinate mapping module, a brightness statistics module, an image gain compensation coefficient calculation module, a pixel gain coefficient calculation module, and an image gain compensation output module. The data module is divided into a data acquisition module and a parameter configuration module.

[0014] The data acquisition module acquires the current frame stereo image data Frame_data1 and Frame_data2 from the real-time video stream, as well as the coordinate mapping files that map the stitched image to the stereo camera source image; leftlut.bin and rightlut.bin or toplut.bin and downlut.bin.

[0015] The parameter configuration module configures the input size DstSize, output size SrcSize, and mapping file size LutSize parameters. The parameter configuration module reads and configures these parameters by reading the frame header content of the coordinate mapping module. This frame header includes the input size DstSize, output size SrcSize, and mapping file size LutSize parameters. The coordinate mapping module calculates the pixel value Value of the stitched image Dst(x,y) in the source image Src(x,y) based on the coordinate mapping obtained by the coordinate mapping module.

[0016] The brightness statistics module obtains sumY by accumulating the pixel values ​​of all images from the stereo camera within the repeating region. If it is a left and right stereo camera, it performs horizontally stitched stereo image gain compensation, that is, it calculates the brightness of leftSumY and rightSumY by using the repeating regions from the left and right images of the stereo stitched image respectively. If it is a top and bottom stereo camera, it performs vertically stitched stereo image gain compensation, that is, it calculates the brightness of topSumY and downSumY by using the repeating regions from the top and bottom images of the stereo camera respectively.

[0017] The image gain compensation coefficient calculation module calculates the gain compensation coefficient gainCorrection of the current frame's stereo image by obtaining the brightness mean values ​​luma_mean1 and luma_mean2 of the previous frame's stereo image. Ratio 1 and gainCorrection Ratio 2 Because the environment of adjacent frames is considered continuous, and for computational efficiency, the brightness information of the previous frame is used to compensate for the next frame; the formula is shown below:

[0018] gainCorrection Ratio 1 =(luma_mean1+luma_mean2) / 2 / luma_mean2)-1;

[0019] gainCorrection Ratio2 =(luma_mean1+luma_mean2) / 2 / luma_mean2)-l;

[0020] Next, calculate the step size gainStep and the increment gainAdd;

[0021] The pixel gain coefficient calculation module obtains the gain coefficient of the current pixel by calculating the ratio between the current pixel and the coordinates of the gain compensation start point or the gain compensation end point.

[0022] The image gain compensation output module calculates the pixel gain of the current stitched image, performs gain compensation on the image within the gain compensation area of ​​the binocular stitched image, and then outputs the result.

[0023] Value = Value * gainRatio.

[0024] The images from the repeating regions after binocular stitching in the system can be approximated as images of the same environment under different exposure configurations.

[0025] The binocular stitched image includes:

[0026] Binocular image stitching: left and right stitching and overlapping areas;

[0027] Binocular stitched images are stitched together vertically and have overlapping areas.

[0028] The module for calculating image gain compensation coefficients further includes:

[0029] Get the average brightness values ​​luma_mean and gain_order;

[0030] Calculate the gain compensation coefficient gainCorrectionRatio; since the selected repeating region has the same number of pixels from the stereo image, leftSumY can be used instead of luma_mean1. The gain ratio coefficient gainRatio of the left image is shown below:

[0031] gainRatio=(leftSumY+rightSumY) / 2 / leftSumY);

[0032] The gain scaling factor gainRatio in the right figure is shown below:

[0033] gainRatio=(leftSumY+rightSumY) / 2 / rightSumY);

[0034] The gain compensation ratios for the two graphs are shown below:

[0035] gainCorrectionRatio=gainRatio-1;

[0036] If the images are spliced ​​vertically, the gain compensation coefficient gainCorrectionRatio for both images can be calculated similarly. Determine if gainCorrectionRatio > threshold. A threshold value of 1 is recommended.

[0037] If not, then

[0038] gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength),

[0039] gainAdd = 1 / gainStep;

[0040] If so, then

[0041] gainStep = 1,

[0042] gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

[0043] In the pixel gain coefficient calculation module, calculating the image gain compensation coefficient further includes: first, selecting vertical stitching compensation or horizontal stitching compensation according to the gain compensation direction;

[0044] Then, based on the gain compensation order, it is selected whether to compensate for the left or top image, or the right or bottom image. Next, the relationship between the distance of the current pixel to the start or end point of the gain compensation area and the gain compensation distance is calculated. To facilitate the hardware to perform gain compensation on the image within the gain compensation area, the floating-point number gainCorrectionRatio is converted to an integer. Based on the previously calculated gain step size gainStep, the gain increment gainAdd for each step, and the base gain ratio baseRatio (a recommended value for baseRatio is 128), the pixel gain compensation coefficient ratio for the current pixel, which is an integer, is calculated.

[0045] Assuming a horizontal stitching of the left image, this relationship is the ratio of the absolute value of the distance from the horizontal coordinate j of the current pixel to the horizontal coordinate gainStartX of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength.

[0046] gaincorrectionlength=gainEndX-gainStartX;

[0047] Assuming a horizontal stitching of the right image, this relationship is the ratio of the absolute value of the distance from the current pixel's horizontal coordinate j to the horizontal coordinate gainEndX of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength.

[0048] gaincorrectionlength=gainEndX-gainStartX;

[0049] Assuming the above image is stitched vertically, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainStartY of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength.

[0050] gaincorrectionlength=gainEndY-gainStartY;

[0051] Assuming a vertical stitching of the image below, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainEndY of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength.

[0052] gaincorrectionlength=gainEndY-gainStartY.

[0053] In the pixel gain coefficient calculation module, the calculation of pixel gain compensation coefficient further includes: obtaining the current pixel coordinates (x, y), the gain compensation order gainOrder, and the gain compensation direction gainDire;

[0054] First, determine if the gain compensation direction gainDire is 0.

[0055] If gain compensation direction gainDire is 0, then check if gain compensation order gainOrder is 0.

[0056] If true, the pixel gain compensation coefficient ratio of the left image is calculated as follows: ratio = (j - gainStartX) / gainStep * gainAdd) / baseRatio) + baseRatio);

[0057] If this is not true, the pixel gain compensation coefficient ratio in the right figure is as follows: ratio = (j - gainEndX) * gainAdd) / mgainStep / baseRatio) + baseRatio);

[0058] If gain compensation direction gainDire is not true, then determine whether gain compensation direction gainOrder is true.

[0059] If this is not true, then the pixel gain compensation coefficient ratio in the following figure is as follows: ratio = (j - gainEndY) * gainAdd) / mgainStep / baseRatio) + baseRatio;

[0060] If true, the pixel gain compensation coefficient ratio in the above figure is as follows: ratio = (j - gainStartY) / gainStep * gainAdd) / baseRatio) + baseRatio.

[0061] This application also relates to an image exposure gain compensation method for binocular stitched images, the method using any of the above-described systems, comprising the following steps:

[0062] S1, Obtain the coordinate mapping file from the stereo stitched image to the stereo camera source image;

[0063] S2, obtain the pixel values ​​of the stereo stitching image based on the coordinate mapping file;

[0064] S3, the mean brightness of the overlapping region of the stereo stitched image corresponding to the source image of the stereo camera;

[0065] S4, obtain the gain compensation coefficients gainCorrectionRation corresponding to the left and right source images of the stereo camera;

[0066] S5, calculate the gain coefficient gainRatio of the current pixel value in the gain compensation region of the binocular stitched image; S6, output the image gain compensation value = value * gainRatio.

[0067] In step S4, the process of calculating the image gain compensation coefficient includes:

[0068] S4.1, obtain the average brightness values ​​luma_mean and gain_order;

[0069] S4.2, Calculate the gain compensation coefficient gainCorrectionRatio;

[0070] S4.3, determine if gainCorrectionRatio>threshold is true? The recommended value for threshold is 1.

[0071] If not, then

[0072] gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength),

[0073] gainAdd = 1 / gainStep;

[0074] If so, then

[0075] gainStep = 1,

[0076] gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

[0077] In step S5

[0078] First, select either vertical or horizontal splicing compensation based on the gain compensation direction.

[0079] Then, based on the gain compensation order, it is selected whether to compensate for the left or top image, or the right or bottom image; next, the relationship between the distance of the current pixel from the start or end point of the gain compensation area and the gain compensation distance is calculated.

[0080] In step S5, the process of calculating the pixel gain compensation coefficient includes:

[0081] S5.1, obtain the current pixel coordinates (x, y) and gain compensation order gainOrder;

[0082] S5.2, Determine if the gain compensation direction gainDire is 0?

[0083] If true, proceed to step S5.3; if false, proceed to step S5.4.

[0084] S5.3, further determine whether the gain compensation order gainOrder is 0?

[0085] If true, then calculate the pixel gain compensation coefficient ratio of the left image: ratio = (j - gainStartX) / gainStep * gainAdd) / baseRatio) + baseRatio);

[0086] If not, then calculate the pixel gain compensation coefficient ratio of the right figure: ratio = (j-gainEndX)*gainAdd) / mgainStep / baseRatio)+baseRatio);

[0087] S5.4, Determine if the gain compensation order gainOrder is 0?

[0088] If not, calculate the pixel gain compensation coefficients shown in the image below.

[0089] ratio=(j-gainEndY)*gainAdd) / mgainStep / baseRatio)+baseRatio;

[0090] If so, calculate the pixel gain compensation coefficient in the above figure.

[0091] ratio=(j-gainStartY) / gainStep*gainAdd) / baseRatio)+baseRatio.

[0092] The beneficial effects of this invention include:

[0093] 1. The brightness of the overlapping area in the binocular stitched image corresponding to the binocular camera image is taken as the compensation benchmark.

[0094] 2. Exposure gain compensation is performed by comprehensively considering both horizontal and vertical image stitching.

[0095] 3. Gradual linear compensation of exposure gain allows for a smoother transition of exposure gain in binocular stitched images. Attached Figure Description

[0096] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0097] Figure 1 This is a flowchart illustrating the functions implemented by each module of the binocular stitched image exposure gain compensation method in this application.

[0098] Figure 2 This is a schematic diagram of the coordinate mapping of the spliced ​​image in this application.

[0099] Figure 3This is a schematic diagram of the left and right stitching and repeating areas of the binocular stitched image in this application.

[0100] Figure 4 This is a schematic diagram of the stitching and repeating areas of the binocular stitched image in this application.

[0101] Figure 5 This is a flowchart for calculating the image gain compensation coefficient in this application.

[0102] Figure 6 This is a flowchart for calculating the pixel gain compensation coefficient in this application.

[0103] Figure 7 This is a schematic diagram illustrating linear gradient compensation based on the horizontal coordinate when stitching binocular images left and right in this application.

[0104] Figure 8 This is a schematic diagram of the left image, which is a real-world example of the binocular camera before stitching together, in this application.

[0105] Figure 9 This is a schematic diagram of the left image, which is a real-world example of the binocular camera before stitching together, in this application.

[0106] Figure 10 This is a schematic diagram of a real-world stitched image where image exposure gain compensation was not performed during binocular stitching in this application.

[0107] Figure 11 This is a schematic diagram illustrating a real-world stitching example of image exposure gain compensation during binocular stitching in this application. Detailed Implementation

[0108] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0109] This invention addresses the issue of varying exposure gains in binocular stitched images by providing an image exposure gain compensation system and method for such images.

[0110] This system can be divided into a data module, a coordinate mapping module, a brightness statistics module, an image gain compensation coefficient calculation module, a pixel gain coefficient calculation module, and an image gain compensation output module. The functions of each module are described below:

[0111] The data module is mainly divided into a data acquisition module and a parameter configuration module:

[0112] Data acquisition module: Acquires the current frame stereo image data Frame_data1 and Frame_data2 from the real-time video stream, as well as the coordinate mapping file that maps the stitched image to the source image of the stereo camera.

[0113] Parameter configuration module: Configure parameters such as input size DstSize, output size SrcSize, and mapping file size LutSize.

[0114] The coordinate mapping module:

[0115] This section calculates the pixel value (Value) of the stitched image Dst(x, y) in the source image Src(x, y) through coordinate mapping. A schematic diagram is shown below. Figure 2 As shown.

[0116] The brightness statistics module:

[0117] This module calculates the current frame's sumY from the repeated regions of the selected stereo images to obtain the leftSumY and rightSumY of the stereo camera images. After stereo stitching, the image from the repeated regions can be approximated as an image of the same environment under different exposure configurations. A schematic diagram of the left-right stitching and repeated regions of the stereo image is shown below. Figure 3 As shown.

[0118] A schematic diagram of the stitching and overlapping areas of binocular image stitching is shown below. Figure 4 As shown.

[0119] The image gain compensation coefficient calculation module:

[0120] This module calculates the gain compensation coefficient (gainCorrection) of the current frame's stereo image by obtaining the mean brightness values ​​(luma_mean1 and luma_mean2) of the previous frame's stereo image. Ratio 1 and gainCorrection Ratio 2 The formula is shown below. Next, calculate the step size gainStep and the increment gainAdd.

[0121] gainCorrection Ratio 1 =(luma_mean1+luma_mean2) / 2 / luma_mean2)-1

[0122] gainCorrection Ratio2 =(luma_mean1+luma_mean2) / 2 / luma_mean2)-1

[0123] The process for calculating image gain compensation coefficients is as follows: Figure 5 As shown:

[0124] Get the average brightness values ​​luma_mean and gain_order;

[0125] Calculate the gain compensation coefficient gainCorrectionRatio;

[0126] Determine if gainCorrectionRatio > threshold. A threshold value of 1 is recommended.

[0127] If not, then

[0128] gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength),

[0129] gainAdd = 1 / gainStep;

[0130] If so, then

[0131] gainStep = 1,

[0132] gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

[0133] The pixel gain coefficient calculation module:

[0134] This module obtains the gain coefficient of the current pixel by calculating the ratio between the current pixel and the coordinates of the gain compensation start point or the gain compensation end point.

[0135] First, select either vertical or horizontal stitching compensation based on the gain compensation direction. Then, select whether to compensate the left or top image, or the right or bottom image, based on the gain compensation order. Next, calculate the relationship between the current pixel's distance from the start or end point of the gain compensation area and the gain compensation distance, and then calculate the pixel gain compensation coefficient process, as follows: Figure 6 As shown.

[0136] Get the current pixel coordinates (x, y) and the gain compensation order gainOrder;

[0137] Is it true that the gain compensation direction gainDire is 0?

[0138] If true, then

[0139] Further determine whether the gain compensation order gainOrder being 0 is valid?

[0140] If true, then calculate the pixel gain compensation coefficient of the left image.

[0141] ratio=(j-gainStartX) / gainStep*gainAdd) / baseRatio)+baseRatio);

[0142] If this is not the case, then calculate the pixel gain compensation coefficient in the right image.

[0143] ratio=(j-gainEndX)*gainAdd) / mgainStep / baseRatio)+baseRatio);

[0144] If not, then determine whether the gain compensation order gainOrder is 0.

[0145] If not, calculate the pixel gain compensation coefficients shown in the image below.

[0146] ratio=(j-gainEndY)*gainAdd) / mgainStep / baseRatio)+baseRatio;

[0147] If so, calculate the pixel gain compensation coefficient in the above figure.

[0148] ratio=(j-gainStartY) / gainStep*gainAdd) / baseRatio)+baseRatio.

[0149] The image gain compensation output

[0150] This module calculates the pixel gain of the current stitched image, performs gain compensation on the image within the gain compensation area of ​​the stereo stitched image, and then outputs the result:

[0151] Value = Value * gainRatio.

[0152] A diagram illustrating linear gradient compensation based on the horizontal axis when stitching binocular images left and right is shown below. Figure 7 As shown.

[0153] Furthermore, this application uses the implementation flow of the method of the described system, as follows: Figure 1 As shown: S1, obtain the coordinate mapping file from the stereo stitched image to the stereo camera source image;

[0154] S2, obtain the pixel values ​​of the stereo stitching image based on the coordinate mapping file;

[0155] S3, the mean brightness of the overlapping region of the stereo stitched image corresponding to the source image of the stereo camera;

[0156] S4, obtain the gain compensation coefficients gainCorrectionRation corresponding to the left and right source images of the stereo camera;

[0157] S5, calculate the gain coefficient gainRatio of the current pixel value in the gain compensation region of the binocular stitched image;

[0158] S6, Image gain compensation output value = value * gainRation.

[0159] In step S4, the process of calculating the image gain compensation coefficient includes:

[0160] S4.1, obtain the average brightness values ​​luma_mean and gain_order;

[0161] S4.2, Calculate the gain compensation coefficient gainCorrectionRatio;

[0162] S4.3, determine if gainCorrectionRatio>threshold is true? The recommended value for threshold is 1.

[0163] If not, then

[0164] gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength),

[0165] gainAdd = 1 / gainStep;

[0166] If so, then

[0167] gainStep = 1,

[0168] gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

[0169] In step S5, the process for calculating the pixel gain compensation coefficient is as follows: Figure 6 As shown:

[0170] S5.1, obtain the current pixel coordinates (x, y) and gain compensation order gainOrder;

[0171] S5.2, Determine if the gain compensation direction gainDire is 0?

[0172] If true, proceed to step S5.3; if false, proceed to step S5.4.

[0173] S5.3, further determine whether the gain compensation order gainOrder is 0?

[0174] If true, the pixel gain compensation coefficient ratio in the left image is as follows: ratio = (j - gainStartX) / gainStep * gainAdd) / baseRatio) + baseRatio);

[0175] If this is not true, then the pixel gain compensation coefficient ratio in the right figure is as follows: ratio = (j - gainEndX) * gainAdd) / mgainStep / baseRatio) + baseRatio);

[0176] S5.4, Determine if the gain compensation order gainOrder is 0?

[0177] If not, the pixel gain compensation ratio in the following figure is shown below.

[0178] ratio=(j-gainEndY)*gainAdd) / mgainStep / baseRatio)+baseRatio;

[0179] If so, the pixel gain compensation coefficient ratio in the above figure is as follows:

[0180] ratio=(j-gainStartY) / gainStep*gainAdd) / baseRatio)+baseRatio.

[0181] The effects of the specific embodiments include:

[0182] The left image shows the actual scene before the binocular cameras were stitched together, for example. Figure 8 As shown.

[0183] The left image shows the actual scene before the binocular cameras were stitched together, for example. Figure 9 As shown.

[0184] Examples of real-world stitching without image exposure gain compensation during binocular stitching. Figure 10 As shown. A real-world stitching illustration of image exposure gain compensation during binocular stitching, for example. Figure 11 As shown.

[0185] In summary, the technical solution of this application is as follows:

[0186] 1. The brightness of the overlapping area in the binocular stitched image corresponding to the binocular camera image is taken as the compensation benchmark.

[0187] 2. Exposure gain compensation is performed by comprehensively considering both horizontal and vertical image stitching.

[0188] 3. Calculate the gain step size and gain increment, and calculate the relationship between the distance between the current pixel and the gain compensation starting point and the gain compensation length to calculate the gain coefficient to progressively and linearly compensate for the exposure gain, so that the stereo stitched image can transition the exposure gain more smoothly.

[0189] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image exposure gain compensation system for binocular stitched images, characterized in that, The system includes: a data module, a coordinate mapping module, a brightness statistics module, an image gain compensation coefficient calculation module, a pixel gain coefficient calculation module, and an image gain compensation output module; The data module is divided into a data acquisition module and a parameter configuration module; The data acquisition module acquires the current frame stereo image data Frame_data1 and Frame_data2 of the real-time video stream, as well as the coordinate mapping file that maps the stitched image to the stereo camera source image. The file is leftlut.bin and rightlut.bin or toplut.bin and downlut.bin. The parameter configuration module configures the input size DstSize, output size SrcSize, and mapping file size LutSize parameters. The parameter configuration module reads and configures the parameters by reading the frame header content of the coordinate mapping module, which includes the input size DstSize, output size SrcSize, and mapping file size LutSize parameters. The coordinate mapping module calculates the pixel value Value of the stitched image Dst(x,y) in the source image Src(x,y) by calculating the coordinate mapping obtained by the coordinate mapping module. The brightness statistics module obtains sumY by accumulating the pixel values ​​of all images from the stereo camera within the repeating region. If it is a left and right stereo camera, it performs horizontally stitched stereo image gain compensation, that is, it calculates the brightness of leftSumY and rightSumY by using the repeating regions from the left and right images of the stereo stitched image respectively. If it is a top and bottom stereo camera, it performs vertically stitched stereo image gain compensation, that is, it calculates the brightness of topSumY and downSumY by using the repeating regions from the top and bottom images of the stereo camera respectively. The image gain compensation coefficient calculation module calculates the gain compensation coefficient gainCorrection of the current frame's stereo image by obtaining the average brightness values ​​luma_mean1 and luma_mean2 of the previous frame's stereo image. Ratio1 and gainCorrection Ratio2 The formula is shown below: gainCorrection Ratio1 =(front_mean1+front_mean2) / 2 / front_mean2)-1; gainCorrection Ratio2 =(front_mean1+front_mean2) / 2 / front_mean2)-1; Next, calculate the step size gainStep and the increment gainAdd; The pixel gain coefficient calculation module obtains the gain coefficient of the current pixel by calculating the ratio between the current pixel and the coordinates of the gain compensation start point or the gain compensation end point. The image gain compensation output module calculates the pixel gain of the current stitched image, performs gain compensation on the image within the gain compensation area of ​​the binocular stitched image, and then outputs the result. Value = Value * gainRatio.

2. The image exposure gain compensation system for binocular stitched images according to claim 1, characterized in that, The images from the repeating regions after binocular stitching in the system can be approximated as images of the same environment under different exposure configurations. The binocular stitched image includes: Binocular image stitching: left and right stitching and overlapping areas; Binocular stitched images are stitched together vertically and have overlapping areas.

3. The image exposure gain compensation system for binocular stitched images according to claim 1, characterized in that, The module for calculating image gain compensation coefficients further includes: Get the average brightness values ​​luma_mean and gain_order; Calculate the gain compensation coefficient gainCorrectionRatio; Since the selected repeating region has the same number of pixels from the stereo images, leftSumY can be used instead of luma_mean1. The gain ratio of the left image is shown below: gainRatio=(leftSumY+rightSumY) / 2 / leftSumY); The gain scaling factor gainRatio in the right figure is shown below: gainRatio=(leftSumY+rightSumY) / 2 / rightSumY); The gain compensation ratios for the two graphs are shown below: gainCorrectionRatio=gainRatio-1; If the images are spliced ​​vertically, the gain compensation coefficient gainCorrectionRatio for the upper and lower images can be calculated similarly. Determine if gainCorrectionRatio>threshold holds true. A threshold value of 1 is recommended. If not, then gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength), gainAdd = 1 / gainStep; If so, then gainStep = 1, gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

4. The image exposure gain compensation system for binocular stitched images according to claim 1, characterized in that, In the pixel gain coefficient calculation module, calculating the image gain compensation coefficient further includes: First, choose between vertical or horizontal splicing compensation based on the direction of gain compensation; Then, based on the gain compensation order, select whether to compensate for the left or top image, or the right or bottom image; next, calculate the relationship between the distance of the current pixel from the start or end point of the gain compensation area and the gain compensation distance. Assuming a horizontal stitching of the left image, this relationship is the ratio of the absolute value of the distance from the horizontal coordinate j of the current pixel to the horizontal coordinate gainStartX of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndX-gainStartX; Assuming a horizontal stitching of the right image, this relationship is the ratio of the absolute value of the distance from the current pixel's horizontal coordinate j to the horizontal coordinate gainEndX of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndX-gainStartX; Assuming the above image is stitched vertically, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainStartY of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndY-gainStartY; Assuming a vertical stitching of the image below, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainEndY of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndY-gainStartY.

5. The image exposure gain compensation system for binocular stitched images according to claim 4, characterized in that, In the pixel gain coefficient calculation module, calculating the pixel gain compensation coefficient further includes: Get the current pixel coordinates (x, y), gain compensation order (gainOrder), and gain compensation direction (gainDire); How to determine if the gain compensation direction gainDire is 0? The gainDire value is the gain compensation direction; 0 represents horizontal gain compensation, and non-zero represents vertical gain compensation. If true, then Further determine whether the gain compensation order gainOrder is 0. A value of 0 represents gain compensation in the left or top image, while a value other than 0 represents gain compensation in the right or bottom image. To facilitate the hardware in performing gain compensation on the image within the gain compensation area, the floating-point number gainCorrectionRatio is converted into an integer. Based on the previously calculated gain step size gainStep, the gain increment gainAdd for each step, and the base gain ratio value baseRatio (which is recommended to be 128), the pixel gain compensation coefficient ratio of the current pixel is calculated as an integer. First, determine if the gain compensation direction gainDire is 0. If true, then determine whether the gain compensation direction gainOrder is 0. If true, the pixel gain compensation ratio in the left image is as follows: ratio=(j-gainStartX) / gainStep*gainAdd) / baseRatio)+baseRatio); If this is not the case, the pixel gain compensation coefficient ratio in the right figure is as follows: ratio=(j-gainEndX)*gainAdd) / mgainStep / baseRatio)+baseRatio); If not, then determine whether the gain compensation direction gainOrder is 0. If this is not the case, then the pixel gain compensation ratio in the following figure is as follows: ratio=(j-gainEndY)*gainAdd) / mgainStep / baseRatio)+baseRatio; If true, the pixel gain compensation ratio in the above figure would be as follows: ratio=(j-gainStartY) / gainStep*gainAdd) / baseRatio)+baseRatio.

6. A method for image exposure gain compensation for binocular stitched images, characterized in that, The method uses any one of the systems described in claims 1-5 and includes the following steps: S1, Obtain the coordinate mapping file from the stereo stitched image to the stereo camera source image; S2, obtain the pixel values ​​of the stereo stitching image based on the coordinate mapping file; S3, the mean brightness of the overlapping region of the stereo stitched image corresponding to the source image of the stereo camera; S4, obtain the gain compensation coefficients gainCorrectionRation corresponding to the left and right source images of the stereo camera; S5, calculate the gain coefficient gainRatio of the current pixel value in the gain compensation region of the binocular stitched image; S6, Image gain compensation output value = value * gainRation.

7. The image exposure gain compensation method for binocular stitched images according to claim 6, characterized in that, In step S4, the process of calculating the image gain compensation coefficient includes: S4.1, obtaining the brightness mean values ​​luma_mean and gain_order; S4.2, Calculate the gain compensation coefficient gainCorrectionRatio; S4.3, Determine whether gainCorrectionRatio>threshold is true? If not, then gainStep=1 / (gaincorrectionRadion*Ratio / gainCorrectionLength), gainAdd = 1 / gainStep; If so, then gainStep = 1, gainAdd=gainCorrectionRatio / gainCorrectionLength*Ratio.

8. The image exposure gain compensation method for binocular stitched images according to claim 6, characterized in that, In step S5 First, select either vertical or horizontal splicing compensation based on the gain compensation direction. Then, based on the gain compensation order, select whether to compensate for the left or top image, or the right or bottom image; next, calculate the relationship between the distance of the current pixel from the start or end point of the gain compensation area and the gain compensation distance. Assuming a horizontal stitching of the left image, this relationship is the ratio of the absolute value of the distance from the horizontal coordinate j of the current pixel to the horizontal coordinate gainStartX of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndX-gainStartX; Assuming a horizontal stitching of the right image, this relationship is the ratio of the absolute value of the distance from the current pixel's horizontal coordinate j to the horizontal coordinate gainEndX of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndX-gainStartX; Assuming the above image is stitched vertically, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainStartY of the starting point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndY-gainStartY; Assuming a vertical stitching of the image below, this relationship is the ratio of the absolute value of the distance from the vertical ordinate j of the current pixel to the vertical ordinate gainEndY of the end point of the gain compensation region to the gain compensation distance gaincorrectionlength. gaincorrectionlength=gainEndY-gainStartY.

9. The image exposure gain compensation method for binocular stitched images according to claim 8, characterized in that, In step S5, the process of calculating the pixel gain compensation coefficient includes: S5.1, obtaining the current pixel coordinates (x, y) and the gain compensation order gainOrder; S5.2, Determine if the gain compensation direction gainDire is 0? If true, proceed to step S5.3; if false, proceed to step S5.

4. S5.3, further determine whether the gain compensation order gainOrder is 0? If true, then calculate the pixel gain compensation coefficient ratio of the horizontally stitched left image: ratio = (j - gainStartX) / gainStep * gainAdd) / baseRatio) + baseRatio); If not, then calculate the pixel gain compensation coefficient ratio of the horizontally stitched right image: ratio = (j - gainEndX) * gainAdd) / mgainStep / baseRatio) + baseRatio); S5.4, Determine if the gain compensation order gainOrder is 0? If not, calculate the pixel gain compensation coefficient ratio for vertically stitching the image below: ratio = (j - gainEndY) * gainAdd) / mgainStep / baseRatio) + baseRatio; If so, calculate the pixel gain compensation coefficient ratio for vertically stitching the above image: ratio = (j - gainStartY) / gainStep * gainAdd) / baseRatio + baseRatio.

Images