A fast white balance method based on adaptive pixel screening

Abstract

The present invention discloses a fast white balance method based on adaptive pixel screening, which first uses RGB grayscale to calculate the "highlight-neutral" map of the original color cast image, and then uses the mean and variance of the image in the RGB channel to calculate its "degree of color cast", and then uses the particle swarm optimization algorithm to reduce the "degree of color cast" as the goal, through iterative optimization, seeks a suitable proportion of "highlight-neutral" pixels to estimate the illumination color, and uses a linear mapping method to correct the input image. In view of the color shift caused by colored illumination, white balance algorithms based on statistical assumptions often apply their assumptions universally to all images, while images that violate their prior assumptions cannot be effectively processed. Therefore, the present invention provides an adaptive white balance method, which does not set special scene feature assumptions, and thus has good adaptability to scenes with different characteristics, especially scenes with a single color.

Description

Technical Field

[0001] The invention relates to a fast white balance method based on adaptive pixel screening, and belongs to the technical field of image processing. Background Art

[0002] According to the camera imaging principle, the brightness and color information in a color image is the light reflected by the ambient light through the surface of an object, captured by the camera's photosensitive device, and the numerical value obtained after digital conversion. In other words, the resulting digital image / video is determined by the light and the reflective characteristics of the object's surface. However, in the process of shooting a color image / video, if the ambient light is non-neutral, that is, the light has a certain color, then the color of the resulting image / video will shift toward the color of the ambient light, commonly known as color cast. For example, if you shoot a piece of white paper in a room with yellow light, the paper in the resulting image will be yellow, not white.

[0003] The purpose of white balance technology is to correct the color cast of the image, that is, to remove the influence of the ambient light color so that the color in the image is consistent with the color in the scene being photographed. Simply put, the white objects in the scene are also white in the photo, and other colored objects also maintain their original hue and saturation. However, since the true color of the scene being photographed is unknown, estimating the color of the lighting at the time of shooting from the color image that has been obtained is itself an ill-posed problem. Therefore, the classic white balance algorithm is mainly based on the following two assumptions: the gray-world assumption and the white-patch assumption. Among them, the gray-world assumption assumes that the average color of all objects in the scene is gray. This assumption is basically reasonable for scenes with rich content changes, but it is no longer valid for single scenes (such as grass, sky, etc.). The white-patch assumption assumes that the color of at least one position (pixel) in the scene is white. This assumption is basically valid for most scenes, but because this assumption uses the maximum value of the pixel grayscale as the white point to estimate the light color, the result is easily interfered by noise. The grayscale world hypothesis and white point hypothesis are both based on the pixel features of the image. Weijer et al. (J. van de Weijer, T. Gevers and A. Gijsenij, "Edge-BasedColor Constancy," in IEEE Transactions on Image Processing, vol. 16, no. 9, pp. 2207-2214, Sept. 2007, doi: 10.1109 / TIP.2007.901808) proposed the gray-edge hypothesis based on the statistics of the first-order derivatives and higher-order derivatives of a large number of images. The gray-edge hypothesis assumes that the color difference of all objects in the image is gray. However, the gray-edge hypothesis algorithm is based on sample statistical features and does not make adaptive parameter optimization and adjustment for differentiated images. It has poor processing effect on images with very few edge areas.

[0004] In order to correct the color shift of the image captured under mixed light sources, patent CN118200748A divides the original image into blocks of different sizes twice, estimates the gain values ​​of multiple first image blocks and second image blocks respectively, and eliminates color cast and achieves white balance by fixing the G channel and correcting the grayscale of the RB channel. Generally, the advantage of fixing the grayscale value of the G channel is to protect the image brightness. However, in low light, in order to obtain satisfactory image details, the image brightness needs to be moderately improved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a fast white balance method based on adaptive pixel screening, which can perform color correction on color images and has good adaptability to scenes with different characteristics. The obtained white balance result can accurately display the true color of the scene.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] A fast white balance method based on adaptive pixel screening comprises the following steps:

[0008] Step 1: for an original image with color cast, the brightness map and absolute color difference map of the original image are calculated using the grayscale data of its RGB channels, and the brightness map and the absolute color difference map are fused to obtain a highlight-neutral map of the original image;

[0009] Step 2, using the grayscale data of the RGB channels of the original image to calculate its color cast degree;

[0010] Step 3, with the goal of minimizing the color cast of the output image, use the particle swarm optimization algorithm to find highlight-neutral pixels for estimating the illumination color; initialize the values ​​of the three particles in the particle swarm optimization algorithm and , optimal particle value And the minimum color cast value ;

[0011] Step 4: For the particle swarm optimization algorithm k Iterations, based on the value of the current first particle , select the first one in the highlight-neutral graph in descending order of highlight-neutral values pixels, denoted as pixel set ; According to the values ​​of the second and third particles respectively and , follow the same steps as above to get the pixel set and ; Use the RGB channel grayscale data of the pixels in the three sets corresponding to the original image to estimate the illumination corresponding to the three particles in the current iteration;

[0012] Step 5: Use the three lights obtained in step 4 , and , use the linear mapping method to correct the original image and get three correction result images;

[0013] Step 6: Three calibration result images obtained in step 5 , and , calculate the color cast degree of each correction result image respectively , and find the minimum color cast ,according to With the current minimum color cast value The size relationship between the two is to update the optimal particle value. , minimum color cast And the optimal correction result diagram ;

[0014] Step 7, determine whether the termination condition of the particle swarm optimization algorithm is met, if so, output the optimal correction result image obtained in step 6 as the final white balance result, otherwise proceed to step 8;

[0015] Step 8: Select the value based on the current optimal particle Update the values ​​of the three particles at the next iteration and ,make And return to step 4 to continue iterating.

[0016] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0017] 1. The present invention designs a pixel-level "highlight-neutral" degree description factor for color images, which is used to screen a certain proportion of pixels to estimate the illumination color.

[0018] 2. In order to adaptively improve the accuracy of screening ratio, the present invention proposes a criterion for measuring the "degree of color cast" of images, and based on the criterion for the degree of color cast, takes reducing the "degree of color cast" as the optimization goal, designs a particle swarm optimization algorithm, and adaptively selects a suitable proportion of "highlight-neutral" pixels for the input image to estimate the illumination color.

[0019] 3. For images with color cast caused by non-white ambient lighting, the present invention proposes an adaptive white balance method, which effectively makes up for the disadvantages of white balance algorithms based on statistical assumptions on specific types of images. For different types of scenes, especially single-color scene images, an effective white balance effect is achieved, avoiding color distortion and overcorrection. In addition, the method has a fast processing speed and is suitable for fast white balance of images, and can also be used for video white balance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a fast white balance method based on adaptive pixel screening of the present invention;

[0021] Figure 2 is a set of color cast images according to an embodiment of the present invention, wherein (a) is an indoor scene image with yellow color cast, (b) is an outdoor scene image with blue color cast, (c) is an outdoor grass image with red color cast, and (d) is an indoor desk image with red color cast;

[0022] Figure 3 The embodiment of the present invention corresponds to Figure 2 White balance results for each item in . DETAILED DESCRIPTION

[0023] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0024] like Figure 1 As shown, the present invention provides an adaptive white balance method for color cast images. First, the "highlight-neutral" image of the original image with color cast is calculated using RGB grayscale. Secondly, the "degree of color cast" of the image is calculated using the mean and variance of the RGB channel. Then, a simplified particle swarm optimization algorithm is used to reduce the "degree of color cast". Through iterative optimization, a suitable proportion of "highlight-neutral" pixels is sought to estimate the illumination color, and a linear mapping method is used to correct the input image. When the optimization algorithm iteration terminates, the image with the smallest "degree of color cast" is used as the output result. Specifically, the following steps are included:

[0025] Step A: Input an image , read the grayscale data of its RGB color space ;

[0026] Step B, according to the following formula, using the grayscale of the RGB space, calculate the V channel map of the HSV space to represent the image brightness:

[0027] ,

[0028] in, x and y are the horizontal and vertical coordinates of the pixels in the image, respectively, and max is the maximum value operation. And calculate the absolute color difference map of each pixel :

[0029] ,

[0030] Among them, min is the minimum value operation.

[0031] Then, fusion and Get the input image The "Highlight-Neutral" graph WAS :

[0032] .

[0033] Step C, calculate the "color cast degree" for the input image CD , the specific formula is:

[0034] ,

[0035] in, and The input images are The maximum and minimum values ​​of the grayscale mean of the RGB channels, and The input images are The specific calculation formula for the maximum and minimum values ​​of the grayscale standard deviation of the RGB channel is as follows:

[0036] ,

[0037] in, H and W They are the number of pixels of the image in the height and width directions respectively, and max and min are the maximum and minimum value operations respectively.

[0038] In addition, the parameters t To control the mean separation and variance separation The specific calculation formula is as follows:

[0039] ,

[0040] In order to limit The value of Within the range, when When the value is greater than 1, .

[0041] Step D, using a simplified particle swarm optimization algorithm to reduce the "color cast" of the image CD " is the goal, and a suitable ratio of "highlight-neutral" pixels is sought to estimate the illumination color. First, the following initial conditions are set for the optimization algorithm:

[0042] 3 initial particle values ;

[0043] The optimal image is the input image, that is, ;

[0044] Minimum color shift value ;

[0045] Optimal ratio ;

[0046] Iterations .

[0047] Step E, based on particle value , obtained from step B WAS In the figure, we select WAS The largest value and pixels, and the obtained pixel sets are recorded as and Then, the illumination color is estimated by using the RGB grayscale of the pixels in the three sets corresponding to the original image. , the formula is as follows:

[0048] ,

[0049] Among them, the subscript Represents the particle label, subscript Represents any one of the RGB channels. To find the averaging factor, its value is equal to The number of pixels in .

[0050] Step F, use the three lighting conditions estimated in step E respectively , and , correct the original input image , and get the three correction result graphs of the current iteration. The specific formula is:

[0051] ,

[0052] Among them, the denominator As the division fidelity term, the value is a very small positive number. At this time, let the number of iterations .

[0053] Step G: for the three correction result images obtained in step F , and , use the "color cast degree" formula in step C to calculate the color cast degree , and is used to update the minimum color cast value, optimal ratio, and optimal image (correction result image) currently encountered. The specific rules are as follows:

[0054] ,

[0055] like ,but ;

[0056] like , it will not be updated.

[0057] Step H, determine whether the algorithm can be terminated, that is, whether the optimization algorithm reaches the termination condition.

[0058] Specifically, if the number of iterationsk Equal to 20, or in the last 3 iterations If no update occurs in step G, the optimization is terminated. As the final output result, the algorithm ends.

[0059] Step I: If the algorithm is not finished, then according to the current optimal particle position , respectively update the positions of the three particles in the particle swarm, the formula is as follows:

[0060] ,

[0061] Among them, the parameters To update the step size, the empirical value is 0.2. At this time, return to step E and continue the optimization iteration.

[0062] Figure 2 (a)- Figure 2 (d) is a set of color cast images caused by colored light, where Figure 2 (a) is an indoor scene picture, in which there are mostly yellow objects and a yellow color cast. Figure 2 (b) is an outdoor scene picture with a blue cast. Figure 2 (c) is an outdoor grass picture with a red cast. Figure 2 (d) is a picture of an indoor desk, which has a red color cast. Figure 3 (a)- Figure 3 (d) is to use the method of the present invention to Figure 2 The results of white balance correction of the image in the experiment show that the present invention can adaptively correct the influence of colored illumination and restore the true color in the image scene. Figure 2 The grass in (c) has a single color, basically green, but the image has a red cast. The result after white balancing is as follows: Figure 3 As shown in (c), the green grass is bright in color and clear in details.

[0063] Based on the same inventive concept, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the aforementioned fast white balance method based on adaptive pixel screening are implemented.

[0064] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the aforementioned fast white balance method based on adaptive pixel screening are implemented.

[0065] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] The present invention is described with reference to flowcharts of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process in the flowchart and the combination of processes in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.

[0067] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0068] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

[0069] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. A fast white balance method based on adaptive pixel screening, characterized in that: The steps include: Step 1: for an original image with color cast, the brightness map and absolute color difference map of the original image are calculated using the grayscale data of its RGB channels, and the brightness map and the absolute color difference map are fused to obtain a highlight-neutral map of the original image; In step 1, the calculation formula for the brightness value of each pixel in the brightness map is as follows: V(x,y)=max{I R (x,y),I G (x,y),I B (x,y)} Among them, V(x,y) represents the brightness value of the pixel (x,y) in the original image, I R (x,y),I G (x,y) and I B (x, y) represents the grayscale value of the pixel (x, y) in the R, G, and B channels of the original image respectively; The absolute color difference calculation formula for each pixel in the absolute color difference map is as follows: CH(x,y)=V(x,y)-min{I R (x,y),I G (x,y),I B (x,y)} Among them, CH(x,y) represents the absolute color difference of pixel (x,y) in the original image; The formula for calculating the highlight-neutral value of each pixel in the highlight-neutral map is as follows: WAS(x,y)=V(x,y)·CH(x,y) Among them, WAS(x,y) represents the combined intensity of brightness and color of pixel (x,y) in the original image; Step 2, using the grayscale data of the RGB channels of the original image to calculate its color cast degree; Step 3, with the goal of minimizing the color cast of the output image, use the particle swarm optimization algorithm to find highlight-neutral pixels for estimating the illumination color; initialize the values ​​of the three particles in the particle swarm optimization algorithm and The optimal particle value is p0% and the minimum color cast value is CD0; Step 4: For the kth iteration of the particle swarm optimization algorithm, based on the value of the current first particle In the highlight-neutral graph, select the top pixels, recorded as pixel set Q1; according to the values ​​of the second and third particles and Follow the same steps as above and select the top highlight-neutral values ​​in the highlight-neutral graph from large to small. and pixels, and obtain pixel sets Q2 and Q3; use the RGB channel grayscale data of the pixels in the three sets corresponding to the original image to estimate the illumination corresponding to the three particles in the current iteration; Step 5: Use the three illuminations (L 1,R ,L 1,G ,L 1,B ),(L 2,R ,L 2,G ,L 2,B ) and (L 3,R ,L 3,G ,L 3,B ), use the linear mapping method to correct the original image and obtain three correction result images; Step 6, for the three correction result graphs (I 1,R ,I 1,G ,I 1,B ),(I 2,R ,I 2,G ,I 2,B ) and (I 3,R ,I 3,G ,I 3,B ), calculate the color cast degree {CD1, CD2, CD3} of each correction result image respectively, and find the minimum color cast degree CD S , according to CD S The relationship between the current minimum color cast value CD0 and the optimal particle value p0%, the minimum color cast value CD0 and the optimal correction result diagram (I 0,R ,I 0,G ,I 0,B ); Step 7, determine whether the termination condition of the particle swarm optimization algorithm is met, if so, output the optimal correction result image obtained in step 6 as the final white balance result, otherwise proceed to step 8; Step 8: Update the values ​​of the three particles in the next iteration according to the current optimal particle value p0% and Let k=k+1 and return to step 4 to continue iterating.

2. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 2, the calculation formula of the color cast degree is as follows: CD(I)=(1-t)(M max -M min )+t(Sd max -Sd min ) Among them, CD(I) represents the color cast of the original image I, M max and M min are the maximum and minimum grayscale mean values ​​of the original image I in the R, G and B channels, respectively, Sd max and Sd min are the maximum and minimum values ​​of the grayscale standard deviation of the original image I in the R, G and B channels respectively. The parameter t represents the weight used to control the mean separation and variance separation. The calculation formula is as follows: t∈[0,1], when the value of t is greater than 1, set t=1.

3. The fast white balance method based on adaptive pixel screening according to claim 2, characterized in that: The maximum and minimum values ​​of the grayscale mean of the original image I in the R, G and B channels are calculated as follows: M max =max{M R ,M G ,M B } M min =min{M R ,M G ,M B } The maximum and minimum values ​​of the grayscale standard deviation of the original image I in the R, G, and B channels are calculated as follows: Sd max =max{Sd R ,Sd G ,Sd B } Sd min =min{Sd R ,Sd G ,Sd B } Among them, I R (x,y),I G (x,y) and I B (x, y) represent the grayscale values ​​of the pixel (x, y) in the original image R, G, and B channels respectively. R ,M G and M B Respectively represent the grayscale mean of the original image I in the R, G and B channels, Sd R ,Sd G and Sd B They represent the grayscale standard deviation of the original image I in the R, G, and B channels respectively, and H and W are the number of pixels in the height and width directions of the original image respectively.

4. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 3, the three particles in the particle swarm optimization algorithm and The initial values ​​of are 1%, 15% and 30% respectively, the initial optimal particle value p0%=1%, and the initial minimum color cast value CD0 is the color cast degree CD(I) of the original image I.

5. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 4, the calculation formula of illumination is as follows: Among them, L r,C represents the value of the illumination estimated by particle r in the C channel, where the subscript r represents the particle label, r∈{1,2,3}, and the subscript C represents any one of the R, G, and B channels, C∈{R,G,B}, n r For the set Q r The number of pixels in I C (x,y) represents the grayscale value of pixel (x,y) in the C channel of the original image.

6. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 5, the original image is corrected using a linear mapping method, and the specific formula is: Among them, I C (x, y) represents the gray value of pixel (x, y) in the C channel of the original image, L r,C It represents the value of the light estimated by particle r in channel C, I r,C (x, y) represents the grayscale value of the pixel (x, y) obtained by particle r correction under the C channel, and ε is the division fidelity term.

7. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 6, the update rules are as follows: If CD S <CD0, then If CD S ≥CD0, no update; Among them, CD S Indicates the minimum value of the color cast corresponding to the three correction result images, p S % represents the particle value corresponding to the minimum color cast degree, (I S,R ,I S,G ,I S,B ) represents the correction result diagram of the particle corresponding to the minimum color cast degree.

8. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 7, the termination condition of the particle swarm optimization algorithm is: reaching a preset number of iterations or starting from the current iteration, the minimum color cast value CD0 is not updated in the previous three iterations.

9. The fast white balance method based on adaptive pixel screening according to claim 1, characterized in that: In step 8, the values ​​of the three particles in the next iteration are updated, and the formula is as follows: in, are the values ​​of particle r at the k+1th and kth iterations respectively, and the parameter s is the update step size.

Images