A color constancy method based on grey surface extraction

Abstract

In order to restore the real color of the object in the image in the human eye due to the illumination color of different ambient light changing the color of the object in the image scene, the application discloses a color constancy algorithm based on gray surface extraction, which extracts the gray pixels in the image based on the illumination invariant vector, then uses the gray world algorithm on the pixels to predict the light source in the image, and corrects the color adaptation of the image. According to the color adaptation result, the gray pixels in the image are continuously detected and the color adaptation is carried out through iterative circulation until the set threshold is met, and the algorithm is stopped. Compared with the traditional algorithm, the illumination estimation accuracy is greatly improved.

Description

Technical Field

[0001] This invention relates to a method for color constancy based on gray surface extraction, belonging to the field of computer vision and image processing technology. Background Technology

[0002] With the rapid development of computer vision and digital image processing, algorithms initially based on grayscale images have been extended to color images. The color information of objects has gradually become a research hotspot, a prerequisite for visual tasks such as feature extraction, object detection, semantic segmentation, and image recognition. However, color information is extremely unstable. In real-world scenarios, on the one hand, the color of an object is determined by the reflective properties of its material; on the other hand, it is easily affected by changes in illumination, causing color shifts in the observed image and interfering with many computer vision tasks and practical applications. Therefore, we need to first restore the image color to that under standard lighting conditions, that is, achieve color constancy. Achieving color constancy is an important component of modern digital camera image preprocessing systems; its purpose is to eliminate the influence of lighting color on the colors of objects in the image scene.

[0003] Color constancy refers to the perceptual characteristic of the human visual system that allows it to consistently perceive the color of an object despite changes in the intensity and spectral distribution of light. Human vision can achieve color constancy through various means, compensating for the impact of changes in lighting color on the appearance of object colors. Achieving computational color constancy in images requires first predicting the true lighting color, and then color adapting the image to eliminate the influence of scene lighting.

[0004] Determining the color of illumination is inherently an underconstrained problem. Therefore, early researchers studying color constancy proposed various assumptions to impose restrictions, designing several classic algorithms based on the statistical color features of the image itself. These algorithms, with their simple computation, low time complexity, and low reproducibility, remain widely used today. However, these assumptions all have limitations. When the image scene to be corrected does not fully satisfy these assumptions, the algorithm's estimation performance becomes unsatisfactory. For example, the gray-world algorithm assumes that all surfaces in the image scene are gray surfaces, but obviously, not all objects in a real scene are gray surfaces. If only gray surfaces in the image could be used for illumination color estimation, it would better align with the algorithm's assumptions. Summary of the Invention

[0005] To address the aforementioned problem of illumination estimation, this invention provides a color constancy method based on gray surface extraction.

[0006] The technical solution adopted in this invention includes the following steps:

[0007] S1. Obtain a color-distorted image as the image to be corrected;

[0008] S2. Perform logarithmic operations on the pixel values ​​of each point in the image to be corrected, and transform the image to the logarithmic domain. Based on this, design an illumination-invariant vector based on the diagonal model.

[0009] S3. Based on the illumination invariant vector, convolve the image using the Laplacian of Gaussian operator to calculate the gray index value of each pixel in the logarithmic domain image. Sort the gray index values ​​from smallest to largest (the smaller the index value, the closer it is to a gray pixel) and filter out a specific proportion of gray surface pixels.

[0010] S4. Select the gray surface pixels filtered in step S3, and use the gray world algorithm to estimate the scene lighting color. Perform color adaptation correction on the original image to be corrected based on the estimated lighting color, and use the corrected image as the new image to be corrected.

[0011] S5. Start the iterative algorithm, calculate the relative deviation values ​​of the R, B, and G channels of each point in the new corrected image obtained in S4, and set a deviation threshold. Filter out pixels with relative deviation values ​​less than the deviation threshold as gray surface pixels. If the number of filtered pixels is less than the set minimum value, use the image obtained in step S4 as the final corrected image; otherwise, use the gray world algorithm to estimate the illumination color of the filtered gray surface pixels, perform color adaptation correction on the new image to be corrected based on the estimated illumination color, and use the corrected image as the image to be corrected for the next processing.

[0012] S6. Continue to a new round of iteration. The processing method is similar to step S5. If the number of selected pixels is less than the set minimum value, the image obtained in step S5 is used as the final corrected image; otherwise, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels.

[0013] S7. Calculate the error between two consecutive illumination estimation results. If the error is less than the set error threshold, stop the algorithm and use the previous corrected image as the final corrected image; otherwise, continue the iterative algorithm. Stop the algorithm when the number of iterations exceeds the set maximum number.

[0014] Furthermore, step S2 includes the following steps:

[0015] S21. First, model the image based on the diagonal model and transform it to the logarithmic domain:

[0016] Based on the process of color digital imaging, the color of all points in the image is represented by the illumination color value I(λ) using the Lambert hypothesis, the surface reflectance of the object S(x, λ), and the sensor response function ρ of the imaging device.c Combinations of (λ):

[0017] f c (x)=∫ ω I(λ)ρ c (λ)S(x,λ)dλ

[0018] Where λ represents the wavelength of the scene light, ω represents the range of the visible spectrum, and x represents the pixel coordinates.

[0019] Assuming the camera's sensitivity function is narrowband, the product of the illumination color value and the imaging device's sensor response function can be considered a constant. Therefore, the formula can be simplified to:

[0020] I i (x, y) = C i (x, y)·R i (x, y), i ∈ {r, g, b}

[0021] r, g, b represent the red, green, and blue components of the image;

[0022] Transform the image to the logarithmic domain:

[0023]

[0024] in, To convert image pixel values ​​to the logarithmic domain, a diagonal model can be used to convert them to the image's illumination component log(C). i (x, y) and reflection component log(R) i The sum of (x, y).

[0025] S22. Design an illumination-invariant vector. The calculation process is as follows: Assuming that the illumination conditions are uniform in a very small local region, the pixel difference between any point (x, y) in the local region of the image and its neighboring pixels can be calculated as follows:

[0026]

[0027] It is the illumination-invariant vector.

[0028] Furthermore, step S3 includes the following steps:

[0029] S31. Convolve the image patch using the Laplacian of Gaussian (LoG) operator to calculate the contrast Δ(x, y) of a 5x5 local region centered on each point in the image:

[0030] △(x, y)=[△ r (x, y), △ g(x, y), △ b (x, y)]

[0031] S32. Design grayscale index values ​​to filter gray surface pixels. Based on the contrast Δ(x, y) obtained in step S31, the gray surface region should satisfy the following as much as possible:

[0032] △ r (x, y) = △ g (x, y) = △ b (x, y) ≠ 0

[0033] S33. Calculate the grayscale index value:

[0034]

[0035] Sort the grayscale index values ​​from smallest to largest, and set a fixed threshold T (range 0-100) to select the top T% of pixels after sorting, set their weight to 1, and discard the other pixels with a weight of 0.

[0036] Furthermore, in step S4, the gray world algorithm calculates the mean value of each channel of all pixels in the image to obtain an illumination color estimation result. Here, it is improved to calculate the mean value of each channel of the points selected in step S3, and then perform color adaptation correction on the image to be corrected based on the diagonal model. The principle of the diagonal model is: after obtaining the illumination color value of the color-biased image, the three channel values ​​of the image are converted into values ​​under standard white light through matrix transformation.

[0037] Furthermore, step S5 includes the following sub-steps:

[0038] S51. Begin the iterative algorithm. Assuming the image obtained after step S4 has been restored to its original state under standard lighting, the three-channel values ​​of the gray surface pixels in the image should theoretically be equal. Therefore, extract the gray surface pixels from the image to be corrected:

[0039]

[0040] S52. Set a threshold τ. Pixels with isGray values ​​of 1 calculated in step S51 are used as gray surface pixels. Filter out all pixels that meet the conditions and count the number length. If the length is less than the set minimum value, stop the iterative algorithm and use the image obtained in step S4 as the final corrected image.

[0041] S53. If the number of pixels (length) is greater than the set minimum value, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels. Based on the estimated illumination color, the new image to be corrected is color-adapted and corrected, and the corrected image is used as the image to be corrected in the next processing.

[0042] Furthermore, in step S7, the illumination estimation results of two adjacent illumination estimations are the illumination color values ​​obtained through steps S6 and S7. When the error is less than a set threshold, it is considered that the estimation result is close enough and there is no need to continue iterating. Therefore, the previous corrected image is selected as the final corrected image; otherwise, the iteration continues until the number of iterations exceeds the set maximum number of iterations.

[0043] This invention discloses a color constancy method based on gray surface extraction. Compared with existing technologies, this invention, based on the traditional gray-world algorithm, further improves the applicability of the assumption by extracting gray surfaces present in the image scene, and proposes a color constancy calculation based on gray surface extraction. This method can more accurately identify gray pixels in the image, obtain a point set for estimating illumination color, and, through iterative algorithms, can extract gray surfaces more accurately. The method proposed in this invention can accurately correct the color of images containing a large number of monochrome regions, which are difficult for gray-world algorithms to handle, and the overall algorithm has higher accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart of the color constancy method based on gray surface extraction according to the present invention.

[0045] Figure 2 This is a flowchart of the gray surface extraction algorithm based on illumination invariant vectors of the present invention. Detailed Implementation

[0046] To better understand the technical solution of the present invention, the process of the present invention will be further described below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, the present invention proposes a color constancy method based on gray surface extraction, comprising the following steps S1 to S7:

[0048] S1. Obtain a color-distorted image as the image to be corrected.

[0049] S2. Perform logarithmic operations on the pixel values ​​of each point in the image to be corrected, and transform the image to the logarithmic domain. Based on this, design an illumination-invariant vector based on the diagonal model.

[0050] Step S2 includes the following sub-steps S21 to S22:

[0051] S21. First, model the image based on the diagonal model and transform it to the logarithmic domain:

[0052]

[0053] in, To convert image pixel values ​​to the logarithmic domain, a diagonal model is used to transform them into the illumination component of the image, log(C). i (x, y) and the reflection component of the object itself in the image log(R) i The sum of (x, y), where i represents any of the red, green, or blue components of the image.

[0054] S22. Design the illumination invariant vector. The calculation process is as follows: Assuming that the illumination conditions are uniform in a very small local area, the illumination component log(C) in the formula is... i (x, y) can be considered a constant. The color difference between any point (x, y) and its neighboring pixels within a local region of the image can be calculated as:

[0055]

[0056] because The change is independent of the illumination component and is determined by the reflection component, therefore... It is called the illumination-invariant vector.

[0057] S3. Based on the illumination invariant vector, convolve the image using the Laplacian of Gaussian operator to calculate the gray index value of each pixel in the logarithmic domain image. Sort the gray index values ​​from smallest to largest and select a specific proportion of gray surface pixels.

[0058] like Figure 2 As shown, step S3 includes the following sub-steps S31 to S33:

[0059] S31. Convolve the image patch using the Laplacian of Gaussian (LoG) operator to calculate the contrast Δ(x, y) of a 5x5 local region centered on each point in the image:

[0060] △(x, y)=[△ r (x, y), △ g (x, y), △ b (x, y)]

[0061] △ r (x, y), △ g (x, y), Δ b (x, y) represent the contrast of the red, green, and blue components in the image pixel values, respectively.

[0062] S32. Based on the contrast Δ(x, y) obtained in step S31, the gray surface area should satisfy the following as much as possible:

[0063] △ r (x, y) = △ g (x, y) = △ b (x, y) ≠ 0

[0064] S33. Design grayscale index values ​​to filter gray surface pixels. The formula for calculating grayscale index values ​​is as follows:

[0065]

[0066] Sort the grayscale index values ​​from smallest to largest, and set a fixed threshold T (range 0-100) to select the top T% of pixels after sorting, set their weight to 1, and discard the other pixels with a weight of 0.

[0067] S4. Select the gray surface pixels filtered in step S3, and use the gray world algorithm to estimate the scene lighting color. Perform color adaptation correction on the original image to be corrected based on the estimated lighting color, and use the corrected image as the new image to be corrected.

[0068] Furthermore, in step S4, the grayscale world algorithm calculates the mean value of each channel of all pixels in the image to obtain an illumination color estimation result. This invention improves this by calculating the mean value of each channel of the points selected in step S3, and then performing color adaptation correction on the image to be corrected based on a diagonal model. The principle of the diagonal model is as follows: after obtaining the illumination color values ​​of the color-biased image, the three channel values ​​of the image are converted to values ​​under standard white light through matrix transformation.

[0069] S5. Start the iterative algorithm, calculate the relative deviation values ​​of the R, B, and G channels of each point in the new image to be corrected obtained in S4, and set a deviation threshold. Filter out pixels with relative deviation values ​​less than the deviation threshold as gray surface pixels. If the number of filtered pixels is less than the set minimum value, the image obtained in step S4 is used as the final corrected image; otherwise, the gray world algorithm is used to estimate the illumination color of the filtered gray surface pixels, and the new image to be corrected is color-adapted based on the estimated illumination color. The corrected image is then used as the image to be corrected for the next processing step.

[0070] Step S5 includes the following sub-steps S51 to S53:

[0071] S51. Begin the iterative algorithm. Assuming the image obtained after step S4 has been restored to its original state under standard lighting, the three-channel values ​​of the gray surface pixels in the image should theoretically be equal. Therefore, extract the gray surface pixels from the image to be corrected:

[0072]

[0073] S52. Set a threshold τ. Pixels whose isGray value calculated in step S51 is less than the threshold τ are used as gray surface pixels. Filter out all pixels that meet the conditions and count the number length. If the length is less than the set minimum value, stop the iterative algorithm and use the image obtained in step S4 as the final corrected image.

[0074] S53. If the number of pixels (length) is greater than the set minimum value, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels. Based on the estimated illumination color, the new image to be corrected is color-adapted and corrected, and the corrected image is used as the image to be corrected in the next processing.

[0075] S6. Continue to a new round of iteration. The processing method is similar to step S5. If the number of selected pixels is less than the set minimum value, the image obtained in step S5 is used as the final corrected image; otherwise, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels.

[0076] In step S6, the processing step (5) is the same.

[0077] S7. Calculate the error between two consecutive illumination estimation results. If the error is less than the set error threshold, stop the algorithm and use the previous corrected image as the final corrected image; otherwise, continue the iterative algorithm. Stop the algorithm when the number of iterations exceeds the set maximum number.

[0078] Furthermore, in step S7, the illumination estimation results of two adjacent illumination estimations are the illumination color values ​​obtained through steps S6 and S7. When the error is less than the set threshold, it is considered that the estimation result is close enough and there is no need to continue iterating. Therefore, the previous corrected image is selected as the final corrected image; otherwise, the iteration continues until the number of iterations exceeds the set maximum number of iterations.

[0079] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for color constancy based on gray surface extraction, characterized in that, It includes the following steps: Step S1: Obtain a color-distorted image as the image to be corrected; Step S2: The pixel value of each point in the image to be corrected is transformed to the logarithmic domain through logarithmic operation. Based on this, an illumination invariant vector is designed based on the diagonal model. Step S3: Based on the illumination invariant vector, the image is convolved using the Laplacian of Gaussian operator to calculate the gray index value of each pixel in the logarithmic domain image. The gray index values ​​are sorted from smallest to largest to select a certain proportion of gray surface pixels. The specific implementation of step S3 includes the following sub-steps: S31 calculates the contrast of a k*k local region centered at each point in the image by convolving the image with the Laplacian of Gaussian operator. : These represent the contrast of the red, green, and blue components in the image pixel values, respectively, with k being a constant. S32 uses grayscale index values ​​to filter gray surface pixels based on contrast. The gray surface area should satisfy: S33, Calculate the grayscale index value: grayscale index value Sort the pixels from smallest to largest, set a fixed threshold T, with a value range of 0-100, which means selecting the top T% of pixels after sorting and setting their weight to 1, while setting the weight of other pixels to 0 and discarding them. Step S4: Select the gray surface pixels filtered in step S3, use the gray world algorithm to estimate the scene lighting color, perform color adaptation correction on the original image to be corrected based on the estimated lighting color, and use the corrected image as the new image to be corrected. Step S5: Start the iterative algorithm, calculate the relative deviation values ​​of the R, B channels and G channels of each point in the new corrected image obtained in S4, and select the pixels with relative deviation values ​​less than the set deviation threshold as gray surface pixels. If the number of selected pixels is less than the set threshold, the image obtained in step S4 is used as the final corrected image; otherwise, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels, and the new image to be corrected is color-adapted based on the estimated illumination color, and the corrected image is used as the image to be corrected for the next processing. Step S6: Continue to the next iteration. The processing method is the same as in step S5. If the number of selected pixels is less than the set minimum value, the image obtained in step S5 is used as the final corrected image. Otherwise, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels; Step S7: Calculate the error value between two consecutive illumination estimation results. If the error value is less than the set error threshold, stop the algorithm and use the previous corrected image as the final corrected image; otherwise, continue the iterative algorithm; stop the algorithm when the number of iterations exceeds the set iteration threshold.

2. The color constancy method based on gray surface extraction according to claim 1, characterized in that: In step S2, the image is modeled based on the diagonal model and then transformed to the logarithmic domain; in, To convert image pixel values ​​to the logarithmic domain, a diagonal model is used to transform them into the illumination components of the image, which are determined by the light source. The reflection component of the object itself in the image The summation of i, where i represents any of the red, green, or blue components of the image; The calculation process for the illumination invariant vector is as follows: Assuming that the illumination conditions are uniform within a very small local region, the illumination component in the formula... It can be considered a constant. The color difference between any point (x, y) and its neighboring pixels within a local region of the image can be calculated as: because The change is independent of the illumination component and is determined by the reflection component, therefore... It is called the illumination-invariant vector.

3. The color constancy method based on gray surface extraction according to claim 1, characterized in that: The grayscale world algorithm described in step S4 calculates the mean value of each channel of the points selected in step S3, and then performs color adaptation correction on the image to be corrected based on the diagonal model. The principle of the diagonal model is: after obtaining the illumination color value of the color-biased image, the three channel values ​​of the image are converted into values ​​under standard white light through matrix transformation.

4. The color constancy method based on gray surface extraction according to claim 1, characterized in that: The specific implementation of step S5 includes the following sub-steps: S51, begin the iterative algorithm. Assuming the image obtained after step S4 has been restored to its original state under standard lighting, the three-channel values ​​of the gray surface pixels in the image should theoretically be equal. Therefore, extract the gray surface pixels from the image to be corrected: S52, set a threshold Calculate Pixels with a value of 1 are treated as gray surface pixels, and their number (length) is counted. If the length is less than a set threshold, the iterative algorithm is stopped, and the image obtained in step S2 is used as the final corrected image. S53, if the number of pixels (length) is greater than the set threshold, the gray world algorithm is used to estimate the illumination color of the selected gray surface pixels, and the new image to be corrected is color-adapted based on the estimated illumination color. The corrected image is then used as the image to be corrected in the next processing step.

5. The color constancy method based on gray surface extraction according to claim 1, characterized in that: In step S7, the illumination estimation results of two adjacent illumination estimations are the illumination color values ​​obtained through steps S6 and S7. When the error is less than the set threshold, it is considered that the estimation result is close enough and there is no need to continue iterating. Therefore, the previous corrected image is selected as the final corrected image; otherwise, the iteration continues until the number of iterations exceeds the set maximum number of iterations.

Images