An automatic white balance method based on white point detection and weight fusion

Abstract

The application provides an automatic white balance method based on white point detection and weight fusion, comprising the following steps: S1, calibrating color temperature Planck curve: S1.1, segmenting an image to obtain image parameters; S1.2, calculating Rgain and Bgain block by block; S2, judging whether each block is a white point; if it is a non-white point, the non-white point block is removed; if it is a white point, step S5 is performed; S3, calculating the color temperature of each white point and giving a corresponding weight; S4, clustering to find locally optimal Rgain and Bgain according to the weight and the falling point; S5, iteratively finding the globally optimal awb gain value in a gradient descent manner. Through the globally optimal solution, the awb gain values calculated from the front and rear image frames of the video stream converge and remain consistent, color jumping between the front and rear frames of the video stream is not easy to occur, and the stability of the video image is better maintained.

Description

Technical Field

[0001] This invention belongs to the field of intelligent video image processing technology, and specifically relates to an automatic white balance method based on white point detection and weight fusion. Background Technology

[0002] In existing technologies, automatic white balance primarily corrects color differences caused by different color temperatures, ensuring that white and neutral color blocks in the image do not exhibit color casts, thus giving the camera the characteristic of color constancy seen by the human eye. Current automatic white balance methods typically obtain color temperature curves by calibrating a series of standard color temperatures, with greater weight given to values ​​closer to the standard curve. Image information is then segmented, statistically analyzed, and weighted to obtain the optimal parameters for automatic white balance. Currently popular methods for using a white reference point for white balance first require converting RGB to YCrCb before locating the white reference point.

[0003] However, existing white balance methods based on white point detection have instability issues when calculating white balance gain. The calculated gain value is a local optimum, not a global optimum. If the calculated AWB gain values ​​of two consecutive frames of a video stream converge differently, color jumps may easily occur between consecutive frames of the video stream.

[0004] In addition, the relevant technical terms are as follows:

[0005] Color temperature: A unit of measurement for the color components in light. When a sensor is imaged under a low color temperature light source, white objects appear reddish, while under a high color temperature light source, white objects appear bluish.

[0006] AWB Gain (Automatic White Balance Gain) plays a crucial role in image processing. AWB Gain corrects white balance by adjusting the gain of the red (R), green (G), and blue (B) channels in the image sensor. Specifically, AWB Gain calculates the average of the RGB channels and then adjusts the gain of each channel using a coefficient k, ensuring that the energy of the three RGB channels remains consistent, thereby restoring the image's true colors. Summary of the Invention

[0007] In order to solve the above problems, the object of the present invention is:

[0008] 1. This invention integrates the white balance method of white point detection and the iterative idea of ​​gray world method. First, the entire image is divided into blocks by the white point detection method to find the gray block regions in the image. The sum of R, G, and B components of the gray block regions is counted. Then, the awb gain value calculated by the white point detection method is obtained. The globally optimal awb gain value is calculated iteratively using this gain value to keep the R, G, and B components of the pixels in the gray area equal.

[0009] 2. Increase color temperature bias by assigning different weights to different color temperatures, which can more effectively restore gray and white pixels in images for mixed color temperature scenes.

[0010] Specifically, this method is an automatic white balance method based on white point detection and weight fusion, the method comprising the following steps:

[0011] S1: Calibrate the Planck curve for color temperature, including:

[0012] S1.1: Segment the image and obtain image parameters, including the width and height of the image and the order of the RGB channels in the image;

[0013] S1.2: Calculate the R / G and B / G ratios block by block. The calculation process is as follows: sum the values ​​of all RGB-accessible pixels in the current block, and then use the sum to calculate R / G and B / G.

[0014] S2: Based on the R / G and B / G ratios calculated for each block in the previous step, determine whether each block is a white point. Blocks falling within the Planck curve area are white points, and blocks falling outside the Planck curve area are non-white points. If a block is a non-white point, remove it. If a block is a white point, proceed to step S5.

[0015] S3: Calculate the color temperature of each white point and assign it a corresponding weight;

[0016] S4: Use a weighted average based on the weights and landing points to find the local optima of Rgain and Bgain;

[0017] S5: Use gradient descent to iteratively find the globally optimal Rgain and Bgain.

[0018] Step S1 includes:

[0019] Images of a 24-color chart were acquired in a darkroom under A, U30, TL84, CWF, D50, and D65 light sources. A represents tungsten filament light, U30 is a warm white fluorescent lamp from the US, TL84 is a narrow-band fluorescent light source from Europe, CWF is a cool white fluorescent lamp from the US, D50 is a light source with a slightly warm color tone, and D65 is a light source with a slightly cool color tone. From the calibration images captured at these six color temperatures, the gray area of ​​the 24-color chart was selected, specifically the bottom row of the 24-color chart, as the standard gray area. The RGB pixel values ​​of the gray area were extracted from the 24-color chart images under each light source. The R / G and B / G mean values ​​of each gray block within the gray area are calculated. R / G represents the ratio of the sum of R channel pixels to the sum of G channel pixels in each gray block, and B / G represents the ratio of the sum of B channel pixels to the sum of G channel pixels in each gray block. The calculation results of six sets of color temperatures are statistically analyzed, and these R / G and B / G mean values ​​are plotted on a polygonal line connecting the ratio points. In order to expand the line segment into a band-like region, the dilation operation in image processing technology is used. A 13*13 all-1 convolution kernel is used to perform a convolution operation with the polygonal line. The final result after dilation is the Planck curve under different color temperatures.

[0020] Step S1 further includes:

[0021] The image is divided into blocks, and the mean R / G and B / G values ​​of each block are calculated. The R / G and B / G values ​​are used to determine whether the current block is a gray area. If the coordinates (R / G, B / G) fall within the Planck curve band, the current block belongs to the gray area. The calculated (R / G, B / G) values ​​of each block are mapped onto the Planck curve. Some points fall inside the Planck curve and are considered gray points, while some points fall outside the Planck curve and are not considered gray points. Gray points and white points usually refer to those points whose RGB channel values ​​(Red R, Green G, Blue B) are equal or very close in the color space. All points that do not belong to gray points need to be deleted, and only gray points are kept.

[0022] Step S2 further includes:

[0023] Points falling within the Planck curve are considered white points. In the coordinate system diagram of the Planck curve strips, the horizontal axis represents the R / G ratio, and the vertical axis represents the B / G ratio. In this curve strip diagram, pixel values ​​within the strips are less than 100, while pixel values ​​in other blank areas are all 255. Based on the previously calculated R / G and B / G values, we obtain the position of this point in the diagram, and then obtain the pixel value at the current position. If it equals 255, it is not a white point; if the pixel value is less than 100, it is a white point.

[0024] Step S3 further includes:

[0025] If it's a white dot, calculate the color temperature of the current block. The color temperature (CT) calculation formula is:

[0026] CT = (R / G) / (B / G) * 256

[0027] Calculate the CT values ​​of the 24-color chart under light sources A, U30, TL84, CWF, D50, and D65 respectively, and assign different weights to each. The weight parameters are set to 32, 64, 128, 256, 512, and 256 respectively.

[0028] Step S4 further includes:

[0029] Each white point can have its color temperature (CT) value calculated, and different weighting coefficients can be set for different color temperatures according to user-defined color temperature preferences.

[0030] First, calculate the sum of the weights of all white points, sumW:

[0031]

[0032] Where N*N represents dividing the image into N*N grids, w i This indicates the weight of the i-th cell; if it's not a white cell, the weight is 0. Then, the local Rgain0 and Bgain0 are calculated using a weighted average:

[0033]

[0034] Where R i Let B represent the mean of the R component of the i-th image patch, where B is the mean of the R component of the i-th image patch. i G represents the mean of the B component of the i-th image patch, where G i Let G represent the mean of the G component of the i-th image block.

[0035] Step S5 further includes:

[0036] We use gradient descent to find the global optimum Rgain and Bgain, taking the local Rgain0 and Bgain0 as initial values, and then calculate the global Rgain1 and Bgain1 using gradient descent.

[0037] Find the maximum value MaxV and the minimum value MinV among Rgain1, Bgain1, and 1. If MinV / MaxV > 255 / 256, stop the iteration; otherwise, perform gradient descent iteration to update Rgain0 and Bgain0.

[0038] If MaxV > Rgain1*257 / 256 is true, then execute.

[0039] Target_r = Rgain1 + (MaxV - Rgain1) * 5 / 8, otherwise

[0040] Target_r=MaxVRgain0=Rgain1*Target_r / Rgain0;

[0041] If MaxV > Bgain1*257 / 256 is true, then execute.

[0042] Target_b = Bgain1 + (MaxV - Bgain1) * 5 / 8, otherwise

[0043] Target_b=MaxVBgain0=Bgain1*Target_b / Bgain0;

[0044] The iteration ends when the condition MinV / MaxV>255 / 256 is met, and the global optimal solution Rgain1 and Bgain1 are finally calculated. AWB gain is composed of Rgain and Bgain, and is the general term for Rgain and Bgain.

[0045] The beneficial effects of this invention are as follows: it overcomes the problems in the prior art, and by calculating the global optimal solution, the AWB gain values ​​calculated for the two consecutive frames of the video stream are closer, ensuring the continuity and naturalness of color performance in the video or image sequence; at the same time, since different weights are set for light sources with different color temperatures, it also has a good effect on mixed light source scenes, and can better compensate for color deviations caused by changes in the color temperature of the light source. Attached Figure Description

[0046] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0047] Figure 1 This is a flowchart illustrating the method described in this application.

[0048] Figure 2 It is a Planck line graph.

[0049] Figure 3 This is a schematic diagram of the Planck curve.

[0050] Figure 4 This is a diagram of a 24-color chart.

[0051] Figure 5 This is a schematic diagram of dividing an image into blocks.

[0052] Figure 6 This is a diagram illustrating the landing points of all segments.

[0053] Figure 7 This is a schematic diagram of removing points that fall outside the Planck curve.

[0054] Figure 8 This is a schematic diagram of the code implementation for step S5 of this method. Detailed Implementation

[0055] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0056] This invention relates to an automatic white balance method based on white point detection and weighted fusion, which fuses a white balance method based on white point detection with an iterative search for the globally optimal AWB gain value in the gray-world method. This invention is a technique in Image Signal Processing (ISP) aimed at automatically adjusting the color balance of an image to make white areas appear more natural and accurate.

[0057] like Figure 1 As shown, to achieve the above objectives, the present invention implements them through the following technical solutions and steps:

[0058] S1: Calibrate the Planck curve for color temperature, including:

[0059] S1.1: Segment the image, obtain image parameters, image width and height, and the arrangement order of RGB channels in the image;

[0060] S1.2: Calculate the R / G and B / G ratios block by block. The calculation process is as follows: sum the values ​​of all RGB-accessible pixels in the current block, and then use the sum to calculate R / G and B / G.

[0061] S2: Based on the R / G and B / G ratios calculated for each block in the previous step, determine whether each block is a white point. Blocks falling within the Planck curve area are white points, and blocks falling outside the Planck curve area are non-white points. If a block is a non-white point, remove it. If a block is a white point, proceed to step S5.

[0062] S3: Calculate the color temperature of each white point and assign it a corresponding weight;

[0063] S4: Use a weighted average based on the weights and landing points to find the local optima of Rgain and Bgain;

[0064] S5: Use gradient descent to iteratively find the globally optimal Rgain and Bgain.

[0065] This invention solves the problem of white balance calculation errors in difficult fields such as monochrome and fields without white blocks; it performs white point judgment by dividing the image into blocks to avoid missing white points; it sorts the blocks that are judged as white points according to their brightness to reduce the probability of incorrect white point judgment; and it performs weighted fusion on all white points to reduce the impact of possible incorrect white point judgment.

[0066] Step S1 further includes:

[0067] Images of a 24-color chart were acquired in a darkroom under A, U30, TL84, CWF, D50, and D65 light sources. The 24-color chart is shown below. Figure 4 As shown, A represents tungsten filament light, U30 represents a warm white fluorescent lamp from the US, TL84 represents a narrow-band fluorescent light source from Europe, CWF represents a cool white fluorescent lamp from the US, D50 represents a light source with a slightly warm color tone, and D65 represents a light source with a slightly cool color tone. In the calibration images taken at the above six color temperatures, the gray area of ​​the 24-color card is selected, i.e., the bottom row of the 24-color card is the standard gray area. The RGB pixel values ​​of the gray area are extracted from the 24-color card images under each light source. The R / G and B / G mean values ​​of each gray patch within the gray area are calculated, where R / G represents the ratio of the sum of R channel pixels to the sum of G channel pixels in each gray patch, and B / G represents the ratio of the sum of B channel pixels to the sum of G channel pixels in each gray patch. The calculation results for the six color temperatures are statistically analyzed, and these R / G and B / G mean values ​​are plotted on a broken line connecting the ratio points, as shown below. Figure 2 As shown, to expand the line segment into a strip-shaped region, a dilation operation from image processing techniques is used. This involves convolving a 13x13 all-1 convolution kernel with the line chart. The final dilated result is shown below. Figure 3 As shown, these are the Planck curves at different color temperatures. Further details include:

[0068] Divide the image into blocks, such as Figure 5 As shown, the mean R / G and B / G values ​​of each block are calculated. These R / G and B / G values ​​are used to determine if the current block is a gray area. If the coordinates (R / G, B / G) fall within the Planck curve's banded area, then the current block belongs to the gray area. The calculated (R / G, B / G) values ​​for each block are mapped onto the Planck curve, as shown below. Figure 6 As shown, some points fall within the Planck curve and are therefore considered gray points, while others fall outside the Planck curve and are not. Gray points and white points typically refer to those points whose RGB channel values ​​(Red R, Green G, Blue B) are equal or very close in the color space. All points that are not gray points need to be deleted, such as... Figure 7 Only gray dots are shown.

[0069] Step S2 further includes:

[0070] Points falling within the Planck curve are considered white points. The Planck curve bands are as follows: Figure 3 As shown, the horizontal axis represents the R / G ratio, and the vertical axis represents the B / G ratio. In this graph, the pixel values ​​within the stripes are less than 100, while the pixel values ​​in the other blank areas are all 255. The point's position is determined based on the previously calculated r / g and b / g values. Figure 3 The position is determined, and then the pixel value at the current position is obtained. If it is equal to 255, it is not a white point; if the pixel value is less than 100, it is a white point.

[0071] During AWB white balance correction, the CT value of each block is calculated separately. Then, the weight coefficient of the current block can be found through the weight list. Then, the local optimum AWB gain value is found by weighted averaging. Finally, the global optimum AWB gain is obtained by gradient descent.

[0072] Step S3 further includes:

[0073] If it's a white point, calculate the color temperature of the current block. The formula for calculating the color temperature (CT) is:

[0074] CT = (R / G) / (B / G) * 256

[0075] Calculate the CT values ​​of the 24-color chart under light sources A, U30, TL84, CWF, D50, and D65 respectively, and assign different weights to each. The weight parameters are set to 32, 64, 128, 256, 512, and 256 respectively.

[0076] Step S4 further includes:

[0077] Each landing point can calculate the color temperature (CT) value, and different weighting coefficients can be set for different color temperatures according to custom color temperature preferences.

[0078] First, calculate the sum of the weights of all white points, sumW:

[0079]

[0080] Where N*N represents dividing the image into N*N grids, w i This indicates the weight of the i-th cell; if it's not a white cell, the weight is 0. Then, the local Rgain0 and Bgain0 are calculated using a weighted average:

[0081]

[0082] Where R i Let B represent the mean of the R component of the i-th image patch, where B is the mean of the R component of the i-th image patch. i G represents the mean of the B component of the i-th image patch, where G iLet G represent the mean of the G component of the i-th image block.

[0083] Step S5 further includes:

[0084] We use gradient descent to find the global optimum Rgain and Bgain, taking the local Rgain0 and Bgain0 as initial values, and then calculate the global Rgain1 and Bgain1 using gradient descent.

[0085] Find the maximum value MaxV and the minimum value MinV among Rgain1, Bgain1, and 1. If MinV / MaxV > 255 / 256, stop the iteration; otherwise, perform gradient descent iterations to update Rgain0 and Bgain0. Figure 8 As shown:

[0086] If MaxV > Rgain1*257 / 256 is true, then execute.

[0087] Target_r = Rgain1 + (MaxV - Rgain1) * 5 / 8, otherwise

[0088] Target_r=MaxVRgain0=Rgain1*Target_r / Rgain0;

[0089] If MaxV > Bgain1*257 / 256 is true, then execute.

[0090] Target_b = Bgain1 + (MaxV - Bgain1) * 5 / 8, otherwise

[0091] Target_b=MaxVBgain0=Bgain1*Target_b / Bgain0;

[0092] The iteration ends when the condition MinV / MaxV>255 / 256 is met, and the global optimal solution Rgain1 and Bgain1 are finally calculated. AWB gain is composed of Rgain and Bgain, and is the general term for Rgain and Bgain.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automatic white balance method based on white point detection and weighted fusion, characterized in that, The method includes the following steps: S1: Calibrate the Planck curve for color temperature, including: S1.1: Segment the image and obtain image parameters, including the width and height of the image and the order of the RGB channels in the image; S1.2: Calculate the R / G and B / G ratios block by block. The calculation process is as follows: sum the values ​​of all RGB-accessible pixels in the current block, and then use the sum to calculate R / G and B / G. S2: Based on the R / G and B / G ratios calculated for each block in the previous step, determine whether each block is a white point. Blocks falling within the Planck curve area are white points, and blocks falling outside the Planck curve area are non-white points. If a block is a non-white point, remove it. If a block is a white point, proceed to step S5. S3: Calculate the color temperature of each white point and assign it a corresponding weight; S4: Use a weighted average based on the weights and landing points to find the local optima of Rgain and Bgain; S5: Use gradient descent to iteratively find the globally optimal Rgain and Bgain.

2. The automatic white balance method based on white point detection and weighted fusion according to claim 1, characterized in that, Step S1 includes: Images of a 24-color chart were acquired in a darkroom under A, U30, TL84, CWF, D50, and D65 light sources. A represents tungsten filament light, U30 is a warm white fluorescent lamp from the US, TL84 is a narrow-band fluorescent light source from Europe, CWF is a cool white fluorescent lamp from the US, D50 is a light source with a slightly warm color tone, and D65 is a light source with a slightly cool color tone. From the calibration images captured at these six color temperatures, the gray area of ​​the 24-color chart was selected, specifically the bottom row of the 24-color chart, as the standard gray area. The RGB pixel values ​​of the gray area were extracted from the 24-color chart images under each light source. The R / G and B / G mean values ​​of each gray block within the gray area are calculated. R / G represents the ratio of the sum of R channel pixels to the sum of G channel pixels in each gray block, and B / G represents the ratio of the sum of B channel pixels to the sum of G channel pixels in each gray block. The calculation results of six sets of color temperatures are statistically analyzed, and these R / G and B / G mean values ​​are plotted on a polygonal line connecting the ratio points. In order to expand the line segment into a band-like region, the dilation operation in image processing technology is used. A 13*13 all-1 convolution kernel is used to perform a convolution operation with the polygonal line. The final result after dilation is the Planck curve under different color temperatures.

3. The automatic white balance method based on white point detection and weighted fusion according to claim 2, characterized in that, Step S1 further includes: The image is divided into blocks, and the mean R / G and B / G values ​​of each block are calculated. The R / G and B / G values ​​are used to determine whether the current block is a gray area. If the coordinates (R / G, B / G) fall within the Planck curve band, the current block belongs to the gray area. The calculated (R / G, B / G) values ​​of each block are mapped onto the Planck curve. Some points fall inside the Planck curve and are considered gray points, while some points fall outside the Planck curve and are not considered gray points. Gray points and white points usually refer to those points whose RGB channel values ​​(red R, green G, blue B) are equal or very close in the color space. All points that are not gray points need to be deleted, and only gray points are kept.

4. The automatic white balance method based on white point detection and weighted fusion according to claim 1, characterized in that, Step S2 further includes: Points falling within the Planck curve are considered white points. In the coordinate system diagram of the Planck curve strips, the horizontal axis represents the R / G ratio, and the vertical axis represents the B / G ratio. In this curve strip diagram, pixel values ​​within the strips are less than 100, while pixel values ​​in other blank areas are all 255. Based on the previously calculated R / G and B / G values, we obtain the position of this point in the diagram, and then obtain the pixel value at the current position. If it equals 255, it is not a white point; if the pixel value is less than 100, it is a white point.

5. The automatic white balance method based on white point detection and weighted fusion according to claim 1, characterized in that, Step S3 further includes: If it's a white dot, calculate the color temperature of the current block. The color temperature (CT) calculation formula is: CT = (R / G) / (B / G) * 256 Calculate the CT values ​​of the 24-color chart under light sources A, U30, TL84, CWF, D50, and D65 respectively, and assign different weights to each. The weight parameters are set to 32, 64, 128, 256, 512, and 256 respectively.

6. The automatic white balance method based on white point detection and weighted fusion according to claim 1, characterized in that, Step S4 further includes: Each white point can have its color temperature (CT) value calculated, and different weighting coefficients can be set for different color temperatures according to user-defined color temperature preferences. First, calculate the sum of the weights of all white points, sumW: Where N*N represents dividing the image into N*N grids, w i This indicates the weight of the i-th cell; if it's not a white cell, the weight is 0. Then, the local Rgain0 and Bgain0 are calculated using a weighted average: Where R i Let B represent the mean of the R component of the i-th image patch, where B is the mean of the R component of the i-th image patch. i G represents the mean of the B component of the i-th image patch, where G i Let G represent the mean of the G component of the i-th image block.

7. The automatic white balance method based on white point detection and weighted fusion according to claim 1, characterized in that, Step S5 further includes: We use gradient descent to find the global optimum Rgain and Bgain, taking the local Rgain0 and Bgain0 as initial values, and then calculate the global Rgain1 and Bgain1 using gradient descent. Find the maximum value MaxV and the minimum value MinV among Rgain1, Bgain1, and 1. If MinV / MaxV > 255 / 256, stop the iteration; otherwise, perform gradient descent iteration to update Rgain0 and Bgain0. If MaxV > Rgain1*257 / 256 is true, then execute. Target_r = Rgain1 + (MaxV - Rgain1) * 5 / 8, otherwise Target_r = MaxV Rgain0=Rgain1*Target_r / Rgain0; If MaxV > Bgain1*257 / 256 is true, then execute. Target_b = Bgain1 + (MaxV - Bgain1) * 5 / 8, otherwise Target_b = MaxV Bgain0=Bgain1*Target_b / Bgain0; The iteration ends when the condition MinV / MaxV>255 / 256 is met, and the global optimal solution Rgain1 and Bgain1 are finally calculated. AWB gain is composed of Rgain and Bgain, and is the general term for Rgain and Bgain.

Images