Method and device for balancing green channel of CMOS image sensor

Abstract

The invention relates to the field of image processing, in particular to a method and device for balancing a green channel of a CMOS image sensor, and the method comprises the following steps: S1, obtaining n test images at different exposure times; s2, compressed images are constructed according to the test image, a first image sequence is constructed according to the exposure time sequence, the image mean value of each compressed image is calculated, and a first image mean value sequence is constructed; according to the method, the information of each G channel in the image sensor is analyzed, the correction parameter file is obtained through calculation according to the gray value of each G channel, and then the target image is corrected according to the correction parameter file, so that compared with a common Bayer image green balance method, the method has the advantages that the efficiency is high, and the cost is low. The spectral response difference caused by the manufacturing process of each G channel is considered, and the problems of detail missing, edge running-in, obvious grid noise and the like in the image can be effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and a device for balancing a green channel of a CMOS image sensor. Background Art

[0002] The Bayer image sensor is a widely used image sensor that uses a Bayer filter array to divide each pixel into different color channels, namely R (red) channel, B (blue) channel, Gr (green-red) channel, and Gb (green-blue) channel.

[0003] However, due to the influence of the current Bayer image sensor manufacturing process, each Gr channel and Gb channel may have a spectral response difference, resulting in grid noise in the image. The current common processing method is to set a threshold to judge the grid noise and replace it with the weighted average of the surrounding pixels. This method may cause problems such as missing image details and edge grinding, and does not combine the spectral response difference between the Gr channel and the Gb channel for analysis, so the processed image still has obvious grid noise, and the effect of correcting the imaging of the Bayer image sensor is average. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a method and device for balancing the green channel of a CMOS image sensor, which solves the technical problems in the prior art of detail loss, edge blending and obvious grid noise after correcting the imaging of the Bayer image sensor.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for balancing a green channel of a CMOS image sensor comprises the following steps:

[0007] S1, obtain n test images at different exposure times;

[0008] S2, construct a compressed image according to the test image, and construct a first image sequence [I1, I2, ..., I n ], calculate the image mean of each compressed image and construct the first image mean sequence [M1, M2, …, M n ];

[0009] S3, delete the first image mean sequence [M1, M2, ..., M n ] the image mean of the compressed images under high light conditions and low light conditions, and construct the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,Ieb ];

[0010] S4, according to the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ]Build the calibration parameter file;

[0011] S5, using the correction parameter file to correct the second image sequence [I ea ,…,I eb ] to calibrate all the test images and generate the images to be tested to construct the image sequence to be tested [I a ,…,I b ] and judge whether the calibration parameter file meets the standard;

[0012] If yes, proceed to step S6;

[0013] If not, adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ], and return to step S4;

[0014] S6. Correct the target image according to the correction parameter file.

[0015] Furthermore, in step S2, the following steps are specifically included:

[0016] S21, extracting the grayscale value of the Gr / Gb channel in each Bayer filter array of each test image;

[0017] S22, constructing a compressed image according to the position and grayscale value of each Gr / Gb channel, and constructing a first image sequence [I1, I2, ..., I n ], I i represents the i-th image;

[0018] S23, calculating the image mean of each compressed image, and constructing a first image mean sequence [M1, M2, ..., M n ],M i Represents the image mean of the i-th compressed image.

[0019] Further, in step S23, the compressed image I is calculated i The image mean M i The specific calculation formula is:

[0020]

[0021] Where N represents the compressed image Ii The number of pixels in the G channel; W and H represent the length and width of the compressed image, respectively, N = W × H; P i (x, y) represents the compressed image I i The grayscale value at position (x, y) in .

[0022] Furthermore, in step S3, the following specific steps are included:

[0023] S31, preset gray threshold interval [Gmin, Gmax];

[0024] S32, retain the first image mean sequence [M1, M2, ..., M n ], and construct a second image mean sequence [M a , …, M b ], and its corresponding exposure time is [Exp a ,…,Exp b ];

[0025] S33, according to the exposure time [Exp a ,…,Exp b ] Select the corresponding test image to construct the second image sequence [I ea ,…,I eb ].

[0026] Furthermore, in step S4, the following specific steps are included:

[0027] S41, select a Gr / Gb channel at any position in the image sensor, and according to the second image sequence [I ea ,…,I ea ] constructs a grayscale sequence [G a , …, G b ], where G i Represents the gray value of the i-th test image at the corresponding position;

[0028] S42, using the second image mean sequence [M a , …, M b ] is the vertical coordinate, the grayscale sequence [G a , …, G b ] is used as the horizontal axis for linear fitting, and the correction parameter slope K is calculated, and the calculation formula is:

[0029]

[0030] In the formula, N1=ba;

[0031] S43, calculating the intercept B according to the correction parameter slope K, and the calculation formula is:

[0032]

[0033] In the formula, G i Represents the grayscale sequence [G a , …, G b ] is the gray value of the i-th channel; M i represents the second image mean sequence [M a , …, M b ];

[0034] S44, constructing a correction slope matrix KG and an intercept matrix BG according to the correction parameter slope K and intercept B of the Gr and Gb channels at all positions in the image sensor. The correction slope matrix KG and the intercept matrix BG are the correction parameter files.

[0035] Furthermore, the correction slope matrix KG and the intercept matrix BG correspond one-to-one to the positions of the Gr and Gb channels in the image sensor;

[0036] Among them, the grayscale value of the non-Gr / Gb channel in the correction slope matrix KG is 1, and the grayscale value of the non-Gr / Gb channel in the intercept matrix BG is 0.

[0037] Furthermore, in step S5, the following specific steps are included:

[0038] S51, preset standard deviation threshold σ th ;

[0039] S52, calculate the image sequence to be tested [I a ,…,I b The standard deviation of the gray value of each image to be tested in ] is calculated as follows:

[0040]

[0041] In the formula, σ i Represents the image sequence to be tested [I a ,…,I b ] is the standard deviation of the grayscale value of the i-th image to be tested; M represents the total number of pixels in the image to be tested; I ij Represents the image sequence to be tested [I a ,…,I b ] is the jth gray value of the i-th image to be tested; μ i Represents the image sequence to be tested [I a ,…,I b ] is the image mean of the i-th image to be tested;

[0042] S53, construct a standard deviation sequence [σ a , …, σ b ];

[0043] S54, in the standard deviation sequence [σ a , …, σ b ] to find the maximum value σ max , judge σ max Is it less than σ th ;

[0044] If yes, it means the calibration parameter file meets the standard;

[0045] If not, it means that the calibration parameter file does not meet the standard, and the process goes to step S55;

[0046] S55, adjust the gray threshold interval [Gmin, Gmax], and adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ].

[0047] Furthermore, in step S55, the following steps are included:

[0048] S551, in the image sequence to be tested [I a ,…,I b ] Select the standard deviation greater than σ th The image to be tested;

[0049] S552, calculating the image mean of the corresponding test image according to the image to be tested;

[0050] S553, marking the grayscale threshold interval [Gmin, Gmax] according to the image mean of the corresponding test image, and marking the two endpoints of the grayscale threshold interval [Gmin, Gmax];

[0051] S554. Select the two adjacent marks with the longest distance in the gray threshold interval [Gmin, Gmax] as the adjusted gray threshold interval [Gmin, Gmax].

[0052] Furthermore, in step S6, the following specific steps are included:

[0053] S61, reading the calibration parameter file;

[0054] S62, reading the target image V;

[0055] S63, using the correction parameter file to perform linear correction on the target image V to obtain a corrected image V1 The calculation formula is:

[0056] V 1 (x,y)=V(x,y)×KG(x,y)+BG(x,y)

[0057] Where V(x, y) represents the gray value at the position (x, y) of the target image; KG(x, y) represents the correction parameter slope K at the position (x, y) in the correction slope matrix KG; BG(x, y) represents the intercept B at the position (x, y) in the intercept matrix BG; V 1 (x, y) represents the corrected image V 1 Gray value at position (x, y);

[0058] S64, output corrected image V 1 .

[0059] The present invention also provides a device for realizing a method for balancing a green channel of a CMOS image sensor, comprising:

[0060] A calibration image acquisition unit, wherein the calibration image acquisition unit is used to acquire test images at different exposure times;

[0061] A calibration image storage unit, wherein the calibration image storage unit is used to store test images at different exposure times;

[0062] A calibration parameter calculation unit, the calibration parameter calculation unit is used to calculate the correction parameter slope K and intercept B according to the test images under different exposure times, and generate a correction parameter file;

[0063] A calibration parameter storage unit, wherein the calibration parameter storage unit is used to store a correction parameter file;

[0064] A data input unit, wherein the data input unit is used to read a target image V;

[0065] A correction processing unit, the correction processing unit is used to correct the target image V according to the correction parameter file to generate a corrected image V 1 ;

[0066] A data output unit, the data output unit is used to output the corrected image V 1 .

[0067] Compared with the prior art, the present invention provides a method and device for balancing the green channel of a CMOS image sensor, which has the following beneficial effects:

[0068] 1. The present invention analyzes the imaging information of each G channel in the image sensor, calculates a correction parameter file according to the grayscale value of each G channel, and then corrects the target image according to the correction parameter file. Compared with the common green balance method of Bayer images, the spectral response difference of each G channel caused by the manufacturing process is taken into account, which can effectively avoid the problems of missing details, edge grinding and obvious grid noise in the image.

[0069] 2. The present invention can determine whether there are test images under high-light conditions or low-light conditions in the second image sequence based on the comparison results of the grayscale value standard deviation and the standard deviation threshold of the image sequence to be tested, and then adjust the test images in the second image sequence to avoid the appearance of test images in the second image sequence that affect the calculation of the correction parameter file and affect the accuracy of the subsequent correction of the target image.

[0070] 3. The present invention calculates the correction slope matrix KG and the intercept matrix BG according to the grayscale of each G channel of the test image, and can obtain the correction parameters of each G channel of the image sensor under different lighting conditions according to the imaging characteristics of each G channel of the image sensor. The grayscale value in the correction slope matrix KG at the non-G channel position is 1, and the grayscale value in the intercept matrix BG is 0, which can effectively avoid the influence of the grayscale value of the non-G channel on the correction slope matrix KG and the intercept matrix BG. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0072] Figure 1 is an overall flow chart of the first embodiment of the present invention;

[0073] Figure 2 is a schematic diagram of a Bayer filter array of the present invention;

[0074] Figure 3 is a schematic diagram of the grayscale threshold interval of the present invention;

[0075] Figure 4 This is one of the schematic diagrams for comparing the effects of the image correction of the present invention;

[0076] Figure 5 This is the second schematic diagram for comparing the effects of the image correction of the present invention;

[0077] Figure 6 It is an overall block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0079] Those skilled in the art can understand that all or part of the steps in the following embodiments can be completed by instructing the relevant hardware through a program, so the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0080] The main reason for grid noise in color images is the imbalance of the green channel of the image sensor, which will introduce uneven grid noise during the de-mosaicing interpolation process. At present, the main processing idea for grid noise is to set a threshold to judge the grid noise and replace it with the weighted average of the surrounding pixels. This method may cause problems such as loss of image details and blurred edges. At the same time, the set threshold may not be applicable to multiple scenes, and the image still has obvious grid noise. On the other hand, the current green balance methods all use the collected Bayer image data feature processing, and do not combine the root cause analysis of the green channel difference. Therefore,

[0081] See also Figure 1 As shown, the purpose of the first embodiment of the present invention is to provide a method for balancing the green channel of a CMOS image sensor, which specifically includes the following steps:

[0082] S1. Obtain n test images at different exposure times. Specifically, in order to ensure that other parameters are consistent when analyzing the brightness difference of each G channel in the image sensor under different radiation amounts, in step S1, the specific process includes the following steps:

[0083] S11, setting a uniform white light source and setting a fixed light source intensity;

[0084] S12, setting n exposure times from short to long under the current light source intensity; specifically, adjusting the exposure time enables the image sensor to obtain images with different radiation amounts under a uniform white light source with a fixed intensity;

[0085] S13, under the white light source of the light source intensity, collecting a test image of the image sensor at each exposure time, specifically, the n exposure times are the shortest exposure time until the exposure time when overexposure occurs. In this embodiment, the shortest exposure time is 0.1 second, and the interval between every two adjacent exposure times is 0.1 second;

[0086] The image sensor contains several Bayer filter arrays, each of which contains information on four channels, namely Gr channel, R channel, Gb channel and B channel. Figure 2 As shown, half of the pixels in each Bayer filter array are green, and red and blue each account for a quarter. Gr represents the green-red channel, and Gb represents the green-blue channel. The purpose is to better utilize the high proportion of pixels in the green channel, optimize the image quality and color reconstruction process, and improve the details and overall visual effect of the image.

[0087] S2, construct a compressed image according to the test image, and construct a first image sequence [I1, I2, ..., I n ], calculate the image mean of each compressed image and construct the first image mean sequence [M1, M2, …, M n ]; Specifically, in order to simplify the processing flow and focus on the analysis and correction of the green channel data without considering the influence of other color channel data, step S2 specifically includes the following steps:

[0088] S21, extracting the grayscale value of the Gr / Gb channel in each Bayer filter array of each test image;

[0089] S22, constructing a compressed image according to the position and grayscale value of each Gr / Gb channel, and constructing a first image sequence [I1, I2, ..., I n ], I i represents the i-th image; specifically, since each Bayer filter array in the Bayer image consists of four color channels, Gr, R, Gb, and B, after removing the information of the Gr / Gb, R, and B channels, the row height and column width of the compressed image are both half of those of the test image;

[0090] S23, calculating the image mean of each compressed image, and constructing a first image mean sequence {M1, M2, ..., M n ],M i represents the image mean of the i-th compressed image; specifically, when correcting the test image in the later stage, the image mean of the compressed image is an important parameter, so before calculating the corrected test image, it is necessary to calculate the image mean of each compressed image, so in step S23, the compressed image I is calculated. i The image mean M i The specific calculation formula is:

[0091]

[0092] Where N represents the compressed image Ii The number of pixels in the G channel; W and H represent the length and width of the compressed image, respectively, N = W × H; P i (x, y) represents the compressed image I i The grayscale value at position (x, y) in .

[0093] It is worth noting that the constructed compressed image has only one grayscale value of the G channel, and there is no need to consider the influence of other color channels, and avoids the influence of adjacent non-G channel pixels, thereby improving the accuracy of the final generated correction parameter file.

[0094] S3, delete the first image mean sequence [M1, M2, ..., M n ] the image mean of the compressed images under high light conditions and low light conditions, and construct the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ]; Specifically, under low light conditions (i.e., when the exposure time is short), each pixel receives less light, which is equivalent to simulating low light conditions. At this time, the noise level of the image sensor is relatively high, which will affect the linear response of the low grayscale segment, making the relationship between the signal and the light intensity no longer linear; under high light conditions (i.e., when the exposure time is long), some pixels may be saturated (i.e., overexposure), which is equivalent to simulating high light conditions. At this time, although the noise level of the image sensor is small, due to the saturation effect, the relationship between the signal and the light intensity may still appear nonlinear, please refer to Figure 3 As shown, in order to avoid the nonlinear part of the relationship between the signal and the light intensity from affecting the image correction, the test images under the high light condition and the low light condition need to be deleted. Therefore, in step S3, the following specific steps are included:

[0095] S31. Preset the grayscale threshold interval [Gmin, Gmax]. Specifically, when presetting the grayscale threshold interval, test it experimentally to finally obtain the grayscale threshold interval [Gmin, Gmax]. Gmin should be selected to be higher than the grayscale value at which the noise level increases significantly, so as to ensure that the data used in the linear fitting process has a high signal-to-noise ratio, and Gmin should be high enough to exclude those nonlinear response parts caused by the high noise level under low light conditions. Gmax should be selected to be lower than the grayscale value at which the image begins to show obvious saturation, so as to ensure the effectiveness of the linear response, and Gmax should be low enough to avoid those nonlinear response parts caused by pixel saturation under high light conditions. In this embodiment, the initial value of Gmin is 30, and the initial value of Gmax is 230.

[0096] S32, retain the first image mean sequence [M1, M2, ..., M n ], and construct a second image mean sequence [M a , …, M b ], and its corresponding exposure time is [Exp a ,…,Exp b ];

[0097] S33, according to the exposure time [Exp a ,…,Exp b ] Select the corresponding test image to construct the second image sequence [I ea ,…,I eb ].

[0098] It should be noted that after constructing the grayscale threshold interval, the test images under high-light conditions and low-light conditions can be deleted. Test images with high noise levels under low-light conditions and test images with saturated grayscale values ​​under high-light conditions can be excluded to avoid affecting the accuracy when the correction parameter file is generated later.

[0099] S4, according to the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ] Construct a correction parameter file; specifically, since the manufacturing process causes differences in the spectral response of each image sensor, it is necessary to perform correction according to the spectral response characteristics of each image sensor, so in step S4, the following specific steps are included:

[0100] S41, select a Gr / Gb channel at any position in the image sensor, and according to the second image sequence [I ea ,…,I eb ] constructs a grayscale sequence [G a , …, G b ], where G i Represents the gray value of the i-th test image at the corresponding position;

[0101] S42, using the second image mean sequence [M a , …, M b ] is the vertical coordinate, the grayscale sequence [G a , …, G b ] is used as the horizontal axis for linear fitting, and the correction parameter slope K is calculated, and the calculation formula is:

[0102]

[0103] In the formula, N1=ba;G i Represents the grayscale sequence [G a , …, G b ] is the gray value of the i-th channel; M i represents the second image mean sequence [M a , …, M b ];

[0104] It should be noted that the first image mean sequence is constructed according to the Gr and Gb channels respectively, and when constructing the vertical coordinate, the first image mean sequence of the corresponding type is selected according to the channel type at the position.

[0105] S43, calculating the intercept B according to the correction parameter slope K, and the calculation formula is:

[0106]

[0107] Specifically, a linear fitting is performed using a fitting straight line equation, and the calculation formula is: Y = KX + B, where Y represents the second image mean sequence [M a , …, M b ]; X represents the grayscale sequence [G a , …, G b ];

[0108] S44, constructing a correction slope matrix KG and an intercept matrix BG according to the correction parameter slope K and intercept B of the Gr and Gb channels at all positions in the image sensor, wherein the correction slope matrix KG and the intercept matrix BG are correction parameter files; specifically, the correction slope matrix KG and the intercept matrix BG correspond one-to-one to the positions of the Gr and Gb channels in the image sensor;

[0109] Among them, the grayscale of the non-Gr / Gb channel in the correction slope matrix KG is 1, and the grayscale value of the non-Gr / Gb channel in the intercept matrix BG is 0; specifically, the correction slope matrix KG and the intercept matrix BG facilitate the subsequent accurate and efficient linear correction processing of each G channel pixel in the image sensor, and the grayscale value of the non-G channel position in the correction slope matrix KG is 1, and the grayscale value in the intercept matrix BG is 0, which means that the grayscale values ​​of the R and B channels will not change, which can ensure targeted adjustment of the G channel in the image while avoiding the grayscale value of the non-G channel being affected.

[0110] It should be noted that, by calculating the correction slope matrix KG and the intercept matrix BG according to the grayscale of each G channel of the test image, the correction parameters of each G channel of the image sensor under different lighting conditions can be obtained according to the imaging characteristics of each G channel of the image sensor, and the grayscale value in the correction slope matrix KG at the non-G channel position is 1, and the grayscale value in the intercept matrix BG is 0, which can effectively avoid the influence of the grayscale value of the non-G channel on the correction slope matrix KG and the intercept matrix BG, so it can effectively improve the accuracy of the image sensor in correcting the target image, and avoid problems such as missing details, edge grinding and obvious grid noise in the image.

[0111] S5, using the correction parameter file to correct the second image sequence [I ea ,…,I eb ] to calibrate all the test images and generate the images to be tested to construct the image sequence to be tested [I a ,…,I b ], and according to the image sequence to be tested [I a ,…,I b ]Judge whether the calibration parameter file meets the standard;

[0112] If yes, proceed to step S6;

[0113] If not, adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ], and returns to step S4; specifically, if the correction file does not meet the standard, it means that step S3 does not delete all the compressed images under high light conditions and low light conditions, that is, the values ​​of Gmin and Gmax in the gray threshold interval [Gmin, Gmax] are not accurate enough, and the retained compressed images contain compressed images under high light conditions or low light conditions, which affects the result of the correction parameter file on the test image. In order to solve the above problem, in step S5, the following specific steps are included:

[0114] S51, preset standard deviation threshold σ th In this embodiment, σ th is 10;

[0115] S52, calculate the image sequence to be tested [I a ,…,I b The standard deviation of the gray value of each image to be tested in ] is calculated as follows:

[0116]

[0117] In the formula, σ iRepresents the image sequence to be tested [I a ,…,I b ] is the standard deviation of the grayscale value of the i-th image to be tested; M represents the total number of pixels in the image to be tested; I ij Represents the image sequence to be tested [I a ,…,I b ] is the jth gray value of the i-th image to be tested; μ i Represents the image sequence to be tested [I a ,…,I b ] is the image mean of the i-th image to be tested;

[0118] S53, construct a standard deviation sequence [σ a , …, σ b ];

[0119] S54, in the standard deviation sequence [σ a , …, σ b ] to find the maximum value σ max , judge σ max Is it less than σ th ;

[0120] If yes, it means the calibration parameter file meets the standard;

[0121] If not, it means that the correction parameter file does not meet the standard, and enters step S55; specifically, when the correction parameter file does not meet the standard, it means that the gray threshold interval [Gmin, Gmax] is not appropriate, and it needs to be adjusted at this time, thereby changing the data range of the linear fitting, thereby affecting the final correction parameter slope K and intercept B, which can improve the green balance effect, reduce grid noise, and improve the overall uniformity and clarity of the image;

[0122] S55, adjust the gray threshold interval [Gmin, Gmax], and adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ]; Specifically, after adjusting the grayscale threshold interval [Gmin, Gmax], the corresponding exposure time [Exp a ,…,Exp b ] also changes, depending on the exposure time [Exp a ,…,Exp b ]Adjust the second image mean sequence[M a , …, M b ] and the second image sequence [I ea ,…,I eb ].

[0123] As a further implementation of this embodiment, in step S55, the following steps are included:

[0124] S551, in the image sequence to be tested [I a ,…,I b ] Select the standard deviation greater than σ th The image to be tested;

[0125] S552, calculating the image mean of the corresponding test image according to the image to be tested;

[0126] S553, marking the grayscale threshold interval [Gmin, Gmax] according to the image mean of the corresponding test image, and marking the two endpoints of the grayscale threshold interval [Gmin, Gmax];

[0127] S554, select the two adjacent marks with the longest distance in the gray threshold interval [Gmin, Gmax] as the adjusted gray threshold interval [Gmin, Gmax]. Specifically, in general, the standard deviation of the high gray segment and the low gray segment is large, and the part between the two is low. Generally, the noise in the high gray segment (the part close to 230) and the low gray segment (the part close to 30) is too large or overexposed, while the part between 30 and 230 generally has low noise and will not be overexposed. If the original gray threshold interval [Gmin, Gmax] contains a high gray segment or a low gray segment, the image sequence to be tested [I a ,…,I b ] has a standard deviation greater than σ th Therefore, the part with larger standard deviation will be deleted to avoid the above situation which may affect the accuracy of the correction parameter file.

[0128] It is worth noting that, based on the comparison results of the grayscale value standard deviation and the standard deviation threshold of the image sequence to be tested, it can be known whether there are test images under high-light conditions or low-light conditions in the second image sequence, and then the test images in the second image sequence are adjusted to avoid the appearance of test images in the second image sequence that affect the calculation of the correction parameter file and affect the accuracy of the subsequent correction of the target image.

[0129] S6, correcting the target image according to the correction parameter file; specifically, in order to eliminate the brightness difference of the G channel, in step S6, the following specific steps are included:

[0130] S61, reading the calibration parameter file;

[0131] S62, reading the target image V;

[0132] S63, using the correction parameter file to perform linear correction on the target image V to obtain a corrected image V 1 The calculation formula is:

[0133] V 1 (x,y)=V(x,y)×KG(x,y)+BG(x,y)

[0134] Where V(x, y) represents the gray value at the position (x, y) of the target image; KG(x, y) represents the correction parameter slope K at the position (x, y) in the correction slope matrix KG; BG(x, y) represents the intercept B at the position (x, y) in the intercept matrix BG; V 1 (x, y) represents the corrected image V 1 Gray value at position (x, y);

[0135] S64, output corrected image V 1 .

[0136] It should be noted that the above method can analyze the information of each G channel in the image sensor, and calculate the correction parameter file according to the gray value of each G channel, and then correct the target image according to the correction parameter file. Compared with the common Bayer image green balance method, it takes into account the spectral response difference of each G channel caused by the manufacturing process, and can effectively avoid the problems of missing details, edge grinding and obvious grid noise in the image. Please refer to Figure 4-Figure 5 As shown, Figure 4 The left image is the target image V, and the right image is the corrected image V 1 , Figure 5 In Figure 5 -a is the target image V, Figure 5 -b is the image corrected by the current common green balance method, Figure 5 -c is the corrected image V after correction in this embodiment 1 , Figure 5 -c The box position is compared to Figure 5 -b, the edge grid noise has been basically eliminated, the transition is more natural, and the details are well preserved.

[0137] See also Figure 6 As shown, the purpose of the second embodiment of the present invention is to provide a device for implementing a method for balancing the green channel of a CMOS image sensor. The device for balancing the green channel of a CMOS image sensor includes:

[0138] A calibration image acquisition unit is used to acquire test images at different exposure times; a calibration image storage unit is used to store the test images at different exposure times; a calibration parameter calculation unit is used to calculate the correction parameter slope K and intercept B according to the test images at different exposure times, and generate a correction parameter file; a calibration parameter storage unit is used to store the correction parameter file; a data input unit is used to read the target image V; a correction processing unit is used to correct the target image V according to the correction parameter file and generate a correction image V 1 ; A data output unit, which is used to output the corrected image V 1 .

[0139] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for balancing the green channel of a CMOS image sensor, characterized in that: The specific steps include: S1, obtain n test images at different exposure times; S2, construct a compressed image according to the test image, and construct a first image sequence [I1, I2, ..., I n ], calculate the image mean of each compressed image and construct the first image mean sequence [M1, M2, …, M n ]; S3, delete the first image mean sequence [M1, M2, ..., M n ] the image mean of the compressed images under high light conditions and low light conditions, and construct the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ]; S4, according to the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ]Build the calibration parameter file; S5, using the correction parameter file to correct the second image sequence [I ea ,…,I eb ] to calibrate all the test images and generate the images to be tested to construct the image sequence to be tested [I a ,…,I b ] and judge whether the calibration parameter file meets the standard; If yes, proceed to step S6; If not, adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ], and return to step S4; S6. Correct the target image according to the correction parameter file.

2. The method for balancing the green channel of a CMOS image sensor according to claim 1, characterized in that: In step S2, the following steps are specifically included: S21, extracting the grayscale value of the Gr / Gb channel in each Bayer filter array of each test image; S22, constructing a compressed image according to the position and grayscale value of each Gr / Gb channel, and constructing a first image sequence [I1, I2, ..., I n ], I i represents the i-th image; S23, calculating the image mean of each compressed image, and constructing a first image mean sequence [M1, M2, ..., M n ],M i Represents the image mean of the i-th compressed image.

3. The method for balancing the green channel of a CMOS image sensor according to claim 2, characterized in that: In step S23, the compressed image I is calculated i The image mean M i The specific calculation formula is: Where N represents the compressed image I i The number of pixels in the G channel; W and H represent the length and width of the compressed image, respectively, N = W × H; P i (x, y) represents the compressed image I i The grayscale value at position (x, y) in .

4. The method for balancing the green channel of a CMOS image sensor according to claim 1, characterized in that: In step S3, the following specific steps are included: S31, preset gray threshold interval [Gmin, Gmax]; S32, retain the first image mean sequence [M1, M2, ..., M n ], and construct a second image mean sequence [M a , …, M b ], and its corresponding exposure time is [Exp a ,…,Exp b ]; S33, according to the exposure time [Exp a ,…,Exp b ] Select the corresponding test image to construct the second image sequence [I ea ,…,I eb ].

5. The method for balancing the green channel of a CMOS image sensor according to claim 1, characterized in that: In step S4, the following specific steps are included: S41, select a Gr / Gb channel at any position in the image sensor, and according to the second image sequence [I ea ,…,I eb ] constructs a grayscale sequence [G a , …, G b ], where G i Represents the gray value of the i-th test image at the corresponding position; S42, using the second image mean sequence [M a , …, M b ] is the vertical coordinate, the grayscale sequence [G a , …, G b ] is used as the horizontal axis for linear fitting, and the correction parameter slope K is calculated, and the calculation formula is: In the formula, N1=ba; S43, calculating the intercept B according to the correction parameter slope K, and the calculation formula is: In the formula, G i Represents the grayscale sequence [G a , …, G b ] is the gray value of the i-th channel; M i represents the second image mean sequence [M a , …, M b ]; S44, constructing a correction slope matrix KG and an intercept matrix BG according to the correction parameter slope K and intercept B of the Gr and Gb channels at all positions in the image sensor. The correction slope matrix KG and the intercept matrix BG are the correction parameter files.

6. The method for balancing the green channel of a CMOS image sensor according to claim 5, characterized in that: The correction slope matrix KG and intercept matrix BG correspond one-to-one to the positions of the Gr and Gb channels in the image sensor; Among them, the grayscale value of the non-Gr / Gb channel in the correction slope matrix KG is 1, and the grayscale value of the non-Gr / Gb channel in the intercept matrix BG is 0.

7. The method for balancing the green channel of a CMOS image sensor according to claim 1, characterized in that: In step S5, the following specific steps are included: S51, preset standard deviation threshold σ th ; S52, calculate the image sequence to be tested [I a ,…,I b The standard deviation of the gray value of each image to be tested in ] is calculated as follows: In the formula, σ i Represents the image sequence to be tested [I a ,…,I b ] is the standard deviation of the grayscale value of the i-th image to be tested; M represents the total number of pixels in the image to be tested; I ij Represents the image sequence to be tested [I a ,…,I b ] is the jth gray value of the i-th image to be tested; μ i Represents the image sequence to be tested [I a ,…,I b ] is the image mean of the i-th image to be tested; S53, construct a standard deviation sequence [σ a , …, σ b ]; S54, in the standard deviation sequence [σ a , …, σ b ] to find the maximum value σ max , judge σ max Is it less than σ th ; If yes, it means the calibration parameter file meets the standard; If not, it means that the calibration parameter file does not meet the standard, and the process goes to step S55; S55, adjust the gray threshold interval [Gmin, Gmax], and adjust the second image mean sequence [M a , …, M b ] and the second image sequence [I ea ,…,I eb ].

8. The method for balancing the green channel of a CMOS image sensor according to claim 7, characterized in that: In step S55, the following steps are included: S551, in the image sequence to be inspected [I a ,…,I b ] Select the standard deviation greater than σ th The image to be tested; S552, calculating the image mean of the corresponding test image according to the image to be tested; S553, marking the grayscale threshold interval [Gmin, Gmax] according to the image mean of the corresponding test image, and marking the two endpoints of the grayscale threshold interval [Gmin, Gmax]; S554. Select the two adjacent marks with the longest distance in the gray threshold interval [Gmin, Gmax] as the adjusted gray threshold interval [Gmin, Gmax].

9. The method for balancing the green channel of a CMOS image sensor according to claim 1, characterized in that: In step S6, the following specific steps are included: S61, reading the calibration parameter file; S62, reading the target image V; S63, using the correction parameter file to perform linear correction on the target image V to obtain a corrected image V 1 The calculation formula is: V 1 (x,y)=V(x,y)×KG(x,y)+BG(x,y) Where V(x, y) represents the gray value at the position (x, y) of the target image; KG(x, y) represents the correction parameter slope K at the position (x, y) in the correction slope matrix KG; BG(x, y) represents the intercept B at the position (x, y) in the intercept matrix BG; V 1 (x, y) represents the corrected image V 1 Gray value at position (x, y); S64, output corrected image V 1 .

10. A device for implementing the method for balancing the green channel of a CMOS image sensor according to any one of claims 1 to 9, characterized in that: include: A calibration image acquisition unit, wherein the calibration image acquisition unit is used to acquire test images at different exposure times; A calibration image storage unit, wherein the calibration image storage unit is used to store test images at different exposure times; A calibration parameter calculation unit, the calibration parameter calculation unit is used to calculate the correction parameter slope K and intercept B according to the test images under different exposure times, and generate a correction parameter file; A calibration parameter storage unit, wherein the calibration parameter storage unit is used to store a correction parameter file; A data input unit, wherein the data input unit is used to read a target image V; A correction processing unit, the correction processing unit is used to correct the target image V according to the correction parameter file to generate a corrected image V 1 ; A data output unit, the data output unit is used to output the corrected image V 1 .

Images