Image illumination estimation method and device based on RGB space gray pixels

Abstract

The embodiment of the invention provides an image illumination estimation method based on RGB space gray pixels. According to the method, from the perspective of estimating illumination by using gray pixels, a new illumination invariant descriptor is introduced into an RGB color space to search for the gray pixels, so that image precision loss caused by color space conversion is avoided, and a gray pixel probability judgment function is defined to improve the illumination estimation precision of an algorithm; the provided illumination estimation method is simple and efficient, has very high illumination estimation precision, can be widely applied to engineering application fields such as camera white balance, image classification, object recognition and target tracking, and has very high practicability.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and particularly to an image illumination estimation method and device based on gray pixels in the RGB space. Background Art

[0002] In computer vision applications, the colors of objects in captured images or videos are also affected by ambient light, and the colors of the same object in the same scene under different light sources vary greatly. Since computer vision systems do not have the unique color constancy perception ability of humans and cannot perceive the true colors of objects, computer vision systems will produce deviations or even incorrect results when processing such images. The purpose of illumination estimation is to estimate the external illumination color during imaging and, based on this, eliminate the influence of illumination changes on the colors of objects in the image, enabling the computer vision system to correctly perceive the true colors of objects. Therefore, researching more effective and practical illumination estimation algorithms has very important practical value for improving the performance of computer vision systems.

[0003] Currently, deep learning-based illumination estimation methods usually aim to simply improve the accuracy of illumination estimation without considering the practicality of the algorithms. Due to their complex training processes, slow algorithm running speeds, and difficulty in being transplanted into hardware, they are often difficult to apply in practice. Therefore, from the perspective of practical applications, researching simple and efficient illumination estimation methods based on the underlying statistical characteristics of images and improving their illumination estimation accuracy to meet the requirements of actual systems is of important research significance and practical value. Summary of the Invention

[0004] The object of the present invention is to overcome the deficiencies of existing illumination estimation method technologies and provide an image illumination estimation method based on gray pixels in the RGB (RGB color mode, red-yellow-blue color mode) space.

[0005] An embodiment of the present invention provides an image illumination estimation method based on gray pixels in the RGB space, including:

[0006] Calculating the normalized local reflectance of an image to obtain an illumination-independent image;

[0007] Detecting gray pixels on the illumination-independent image according to the gray pixel probability function;

[0008] Performing illumination estimation on the detected gray pixels.

[0009] In some embodiments, the calculating the normalized local reflectance of an image to obtain an illumination-independent image includes:

[0010] Step S1: Divide the image into k non - overlapping small blocks of the same size, and normalize the pixel values of all pixels within each small block, and calculate the normalized local reflectance L c ,k(x,y).

[0011] In some embodiments, detecting gray pixels on the illumination - invariant image according to the gray pixel probability function includes:

[0012] Step S2: According to the gray pixel probability function DI * (x,y), combined with the normalized local reflectance, detect the gray pixels GT n .

[0013] In some embodiments, estimating the illumination for the detected gray pixels includes:

[0014] Step S3: Select the top n% of the pixels with the highest DI * values in the image as gray pixels, and use these pixels for illumination estimation.

[0015] In some embodiments, in step S1, dividing the image into k non - overlapping small blocks of the same size and normalizing the pixel values of all pixels within each small block, the specific mathematical description is as shown in Equation (1):

[0016]

[0017] In the formula, I c,k (x,y) represents the pixel within the k - th local block, I c,k (x k ,y k ) represents the pixel with the largest pixel value within the k - th local block, L c,k (x,y) is the intensity value of the pixel at (x,y) after normalization within the k - th local block;

[0018] The size of the k local blocks divided is set to 3×3 pixel size. According to Equation (1), the above formula can be rewritten as shown in Equation (2):

[0019]

[0020] By calculating the normalized local reflectance, an illumination - invariant image in the RGB color space is obtained.

[0021] In some embodiments, in step S2, the mathematical definition of the gray pixel probability function is as shown in Equation (4):

[0022]

[0023] Wherein, DI(x,y) represents the probability that a pixel is a gray pixel. is the average value of C for three channels rgb (x,y).

[0024] In some embodiments, considering the influence of image noise and bright pixels on the accuracy of light estimation, isolated gray pixels and pixels with relatively low brightness values in the RGB color space in the image are removed, and the optimized gray pixel probability function is as shown in Equation (5):

[0025]

[0026] Wherein, DI* is the optimized gray pixel probability function, AF η {} is a mean filter with a pixel size of η×η. is the average value of three I's in the RGB color space rgb (x,y), representing the brightness value of a pixel in the RGB color space.

[0027] In some embodiments, in step S3, the top n% of pixels with the highest DI * values in the image are selected as gray pixels, and light estimation is performed using these pixels, including:

[0028] Select the top n% of pixels with the highest DI* values in the image as gray pixels, and define the set of these pixels as GT n , and light estimation is performed using these pixels, and the estimated light e c is defined as shown in Equation (6):

[0029]

[0030] Wherein, N G is the number of pixels in the set GT of gray pixels used to estimate light n , which varies with the resolution of the image.

[0031] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by one or more processors, the method described in any of the above embodiments is implemented.

[0032] An embodiment of the present invention provides an electronic device, including: one or more processors;

[0033] A storage device, on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the method described in any of the above embodiments is implemented.

[0034] The beneficial effects of the above embodiments include:

[0035] Starting from the perspective of estimating illumination using gray pixels, the present invention introduces a new illumination-invariant descriptor in the RGB color space to find gray pixels, avoiding the loss of image accuracy caused by color space conversion, and defines a gray pixel probability determination function to improve the accuracy of illumination estimation in the algorithm. The proposed illumination estimation method is simple and efficient, with high illumination estimation accuracy, and can be widely applied to engineering application fields such as camera white balance, image classification, object recognition, and target tracking, having strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings generally illustrate the various embodiments discussed herein by way of example and not limitation.

[0037] Figure 1 It is a schematic flow chart of the steps of an image illumination estimation method based on gray pixels in the RGB space according to an embodiment of the present invention;

[0038] Figure 2 It is a relationship diagram of the angular error and the gray pixel ratio n% of the present invention on three data sets;

[0039] Figure 3 It is a relationship diagram of the angular error and the filter size η×η of the present invention on three data sets. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] In order to understand the features and technical content of the embodiments of the present application in more detail, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not used to limit the embodiments of the present application.

[0041] In the description of the embodiments of the present application, it should be noted that unless otherwise specified and limited, the term "connection" should be understood in a broad sense. For example, it can be an electrical connection or the communication inside two components. It can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meaning of the above terms can be understood according to specific situations.

[0042] It should be noted that the terms "first / second / third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed. It should be understood that the objects distinguished by "first / second / third" can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here.

[0043] First, calculate the illumination-invariant descriptor in the RGB color space of the image, i.e., the normalized local reflectance, so as to eliminate the influence of illumination on the color of gray pixels without performing color space conversion. To eliminate the influence of illumination on the color of gray pixels and obtain an illumination-independent image. Then, use the defined gray pixel probability function to find gray pixels in the illumination-independent image. Finally, perform illumination estimation using the detected gray pixels. Specifically:

[0044] An image illumination estimation method based on gray pixels in the RGB space according to an embodiment of the present invention is as Figure 1 shown, and the specific steps are as follows:

[0045] Step S1: Divide the image into k non-overlapping small blocks of the same size, and normalize the pixel values of all pixels within each small block to calculate the normalized local reflectance L c ,k(x,y);

[0046] Step S2: According to the gray pixel probability function DI * (x,y), combined with the normalized local reflectance, detect the gray pixels GT n ;

[0047] Step S3: Select the top n% of the pixels with the highest DI * values in the image as gray pixels, and use these pixels for illumination estimation.

[0048] Furthermore, in Step S1, divide the image into k non-overlapping small blocks of the same size, and normalize the pixel values of all pixels within each small block. The specific mathematical description is as shown in Equation (1):

[0049]

[0050] In the formula, I c,k (x,y) represents the pixel within the kth local block, and I c,k (x k ,y k ) represents the pixel with the largest pixel value within the kth local block. Obviously, L c,k (x,y) is the intensity value of the pixel at (x,y) after normalization within the kth local block. In this method, the size of the k divided local blocks is set to 3×3 pixel size. According to Equation (1), the above formula can be rewritten as shown in Equation (2):

[0051]

[0052] As can be seen from the above formula, the normalized local reflectance Lc,k(x,y) is only related to the reflectance of the scene and has nothing to do with the scene illumination. By calculating the normalized local reflectance, an illumination-independent image in the RGB color space can be obtained, so it can be used as an illumination-invariant descriptor in the RGB color space. After obtaining the illumination-independent image in the RGB color space, gray pixels can be further detected and used to estimate the illumination.

[0053] Furthermore, in step S2, since gray pixels have equal reflectance in the three RGB color channels and do not exist in isolation. If a pixel is a gray pixel, then its normalized local reflectance in the three color channels of the RGB space should be equal, and the mathematical description is shown in Equation (3):

[0054] L r (x,y) = L g (x,y) = L b (x,y) (13)

[0055] Theoretically, gray pixels should satisfy Equation (3), but gray pixels cannot be accurately detected only based on Equation (3). There are two reasons: on the one hand, pixels that satisfy Equation (3) are not necessarily gray pixels. For example, the 24 small color patches in the standard color card all satisfy this equation, but they are not all gray; on the other hand, due to the use of an ideal imaging model, the actual image imaging is affected by the camera and the environment, and these interferences will also cause gray pixels not to satisfy Equation (3).

[0056] To solve this problem, this method defines a gray pixel probability function to judge the proximity of a pixel to a gray pixel. The mathematical definition of the gray pixel probability function is shown in Equation (4):

[0057]

[0058] In the formula, DI(x,y) represents the probability that a pixel is a gray pixel, is the average value of C rgb (x,y) of the three channels. When a pixel is more similar to a gray pixel, and the difference between C rgb (x,y) will be smaller. Therefore, the DI value of gray pixels or approximate gray pixels will be higher than that of other pixels.

[0059] Considering the influence of image noise and bright pixels on the accuracy of illumination estimation, isolated gray pixels and pixels with low brightness values in the RGB color space in the image should be excluded. The optimized gray pixel probability function is shown in Equation (5):

[0060]

[0061] In the formula, DI* is the optimized gray pixel probability function, and AF η {} is a mean filter with a pixel size of η×η. is the average of the three I in the RGB color space rgb (x, y), representing the brightness value of a pixel in the RGB color space. It can be seen from this formula that the higher the DI* value of a pixel, the more likely it is to be a gray pixel or an approximate gray pixel.

[0062] Furthermore, in step S3, the gray pixel probability function DI* can be used to measure the probability that each pixel in the image is a gray pixel. The higher the DI* value of a pixel, the closer it is to a gray pixel. In the method, the top n% of the pixels with the highest DI* values in the image (defining the set of these pixels as GT n ) are selected as gray pixels, and these pixels are used for illumination estimation. The estimated illumination e c is defined as shown in formula (6):

[0063]

[0064] In the formula, N G is the set of gray pixels GT used to estimate the illumination n The number of pixels in it varies with the resolution of the image.

[0065] Combined with the appendix Figure 2 、 3 The method for selecting parameters in the method of the present invention is further described as follows:

[0066] The performance of the method of the present invention is affected by two parameters: one is the gray pixel ratio n%, and the other is the mean filter size η×η. Among them, the gray pixel ratio n% is the key parameter of the algorithm, which determines the main performance of the algorithm. And by optimizing the filter size, the illumination estimation accuracy of the algorithm can be further improved. In order to determine the optimal values of these two parameters, the proposed algorithm was tested on three data sets, and the mean and median of the angular error of illumination estimation were used as evaluation indicators. The three data sets for the experiment are the Gehler-Shi image set, the SFU Grey-ball image set, and the SFU laboratory image set.

[0067] 1) Selection of the gray pixel ratio n% parameter

[0068] In order to obtain the optimal value of the gray pixel ratio n% of this method, the idea of controlling variables was adopted. First, the filter size was set to the common 3×3 size, and different ratios of n% were selected to test the algorithm on three data sets. The relationship between the angular error and the gray pixel ratio n% on the three data sets is shown in the appendix Figure 2As shown. It can be seen from the figure that when the gray pixel ratio n% varies from 0.01% to 10%, this method can achieve relatively good illumination estimation results. In addition, when n% = 1%, the median and mean errors of this method on the three datasets are relatively small. For the convenience of the practical application of the algorithm, the gray pixel ratio n% = 1% is taken as the optimal parameter of this method.

[0069] 2) Selection of filter size η×η parameter

[0070] To verify the influence of the filter size on the illumination estimation accuracy, a similar experiment was also carried out. At this time, the gray pixel ratio n% = 1% was directly set. The relationship between the angular error on the three datasets and the filter size η is shown in the appendix Figure 3 As shown. It can be seen from the figure that although the influence of the filter size on the illumination estimation accuracy is not as significant as that of the gray pixel ratio, it still has an important impact on the results. When the filter size η×η is 5×5 pixels, the median and mean errors of this method on the three datasets are the smallest.

[0071] Therefore, finally, the gray pixel ratio n% = 1% and the filter size 5×5 are taken as the optimal parameters of this method.

[0072] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by one or more processors, the method of any of the above embodiments is implemented.

[0073] An embodiment of the present invention provides an electronic device, including: one or more processors;

[0074] A storage device, on which one or more programs are stored. When the one or more programs are executed by one or more processors, the method of any of the above embodiments is implemented.

[0075] Among the technical solutions described in the embodiments of the present application, they can be arbitrarily combined without conflict.

[0076] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for image illumination estimation based on RGB space gray pixels, characterized in that: include: Calculate the normalized local reflectance of the image to obtain an illumination-independent image; Detecting gray pixels on the illumination-independent image according to the gray pixel probability function; Perform lighting estimation on the detected gray pixels.

2. The image illumination estimation method based on RGB space gray pixels according to claim 1, characterized in that: The step of calculating the normalized local reflectivity of the image to obtain the illumination-independent image includes: Step S1: divide the image into k non-overlapping small blocks of the same size, and normalize the pixel values ​​of all pixels in each small block to calculate the normalized local reflectivity L c ,k(x,y).

3. The image illumination estimation method based on RGB space gray pixels according to claim 2, characterized in that: The step of detecting gray pixels on the illumination-independent image according to the gray pixel probability function comprises: Step S2: according to the gray pixel probability function DI * (x, y), combined with the normalized local reflectivity, detect the gray pixel GT in the image n .

4. The image illumination estimation method based on RGB space gray pixels according to claim 3, characterized in that: The step of performing illumination estimation on the detected gray pixels comprises: Step S3, select DI in the image * The top n% pixels with the highest values ​​are treated as gray pixels and used for lighting estimation.

5. The image illumination estimation method based on RGB space gray pixels according to claim 2, characterized in that: In step S1, the image is divided into k non-overlapping small blocks of the same size, and the pixel values ​​of all pixels in each small block are normalized. The specific mathematical description is shown in formula (1): In the formula, I c,k (x, y) represents the pixel in the kth local block, I c,k (x k ,y k ) represents the pixel with the largest pixel value in the kth local block, L c,k (x, y) is the normalized intensity value of the pixel at (x, y) in the kth local block; The size of the k local blocks is set to 3×3 pixels. According to equation (1), the above equation can be rewritten as shown in equation (2): By calculating the normalized local reflectance, an illumination-independent image in the RGB color space is obtained.

6. The image illumination estimation method based on RGB space gray pixels according to claim 3, characterized in that: In step S2, the gray pixel probability function is mathematically defined as shown in formula (4): In the formula, DI(x,y) represents the probability that a pixel is a gray pixel. It is the C of three channels rgb The mean of (x,y).

7. The image illumination estimation method based on RGB space gray pixels according to claim 6, characterized in that: Considering the influence of image noise and bright pixels on the accuracy of illumination estimation, isolated gray pixels and pixels with low brightness values ​​in the RGB color space of the image are removed, and the optimized gray pixel probability function is obtained as shown in formula (5): Where DI* is the optimized gray pixel probability function, AF η {} is a mean filter of size η×η pixels, It is the three I in RGB color space rgb The average value of (x,y) represents the brightness value of a pixel in the RGB color space.

8. The image illumination estimation method based on RGB space gray pixels according to claim 5, characterized in that: In step S3, select DI in the image * The top n% pixels with the highest values ​​are taken as gray pixels and used for lighting estimation, including: Select the first n% pixels with the highest DI* values ​​in the image as gray pixels, and define the set of these pixels as GT n , and use these pixels to estimate the lighting. The estimated lighting e c The definition is shown in formula (6): Where N G It is a set of gray pixels GT used to estimate the lighting n The number of pixels in varies with the resolution of the image.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device having one or more programs stored thereon, which implements the method according to any one of claims 1 to 8 when the one or more programs are executed by the one or more processors.

Images