Vignetting correction method for color intensity entropy and gradient sparsity optimization

Abstract

Vignetting correction is a necessary process for image signal processing and is an important guarantee for obtaining high-quality images. However, the current mainstream vignetting correction method depends on calibration, the workload is large, the reusability is poor, and huge manpower and material resources are consumed in practical application. When the difference between an application scene and a calibration scene is large, the vignetting correction effect is often difficult to guarantee. According to the vignetting correction method based on color intensity entropy and gradient sparsity optimization, by analyzing a vignetting image degradation mechanism, priori knowledge that a non-vignetting image is low in color intensity entropy and sparse in gradient is provided, a vignetting optimization model is constructed accordingly, and a gradient descent method is designed to optimize vignetting parameters. And vignetting correction of the image is realized. The method does not need calibration, is high in accuracy and universality, and is of great significance for promoting landing of a vignetting correction algorithm.

Description

Technical Field

[0001] The invention relates to a vignetting correction method with optimized color intensity entropy and gradient sparsity. Background Art

[0002] Vignetting correction is an essential process for image signal processing and an important guarantee for obtaining high-quality images. The current mainstream methods mainly include calibration method and single-image correction method. Among them, the calibration method has a large workload and poor reusability. Especially when the shooting scene or shooting mode changes, the vignetting effect will also change, resulting in huge calibration costs in industrial applications. The current methods based on single-image correction are generally greatly affected by the scene, the prior assumptions are relatively harsh, the vignetting removal effect is very poor, and it is difficult to combine them into practical applications.

[0003] Therefore, developing a vignetting correction algorithm with strong universality and high accuracy has important research significance at present, and is an important guarantee for obtaining high-quality images. The present invention proposes a vignetting correction method with optimized color intensity entropy and gradient sparsity. Summary of the invention

[0004] The present invention proposes a vignetting correction method for optimizing color intensity entropy and gradient sparsity, so as to solve the above-mentioned technical problems existing in the prior art.

[0005] According to one aspect of the present invention, a vignetting correction method for optimizing color intensity entropy and gradient sparsity is provided, and the process is as follows: Figure 1 As shown, the following steps are included:

[0006] A) according to the polynomial formula of the vignetting model, the default parameter P_0 is set to construct the initial vignetting model V_0, and the vignetting model is used to perform initial correction on the image to obtain the corrected image I_c;

[0007] B) converting the corrected image I_c from RGB space to HSV space to obtain image I_h;

[0008] C) Divide the histogram by color (HS) and intensity (V) values, and perform histogram statistics on the image I_h in the HSV space to obtain the histogram C_H;

[0009] D) the histogram C_H obtained in step C) is processed according to the color dimension, and the probability of the pixel intensity distribution under each color is estimated using the information entropy formula, and the color intensity entropy index E_o of the whole image is obtained accordingly;

[0010] E) using the gradient function to solve the gradient of the image I_c in step C), and solving the L1 norm L_o of the image gradient;

[0011] F) constructing a vignetting correction loss function according to the color intensity entropy index E_o and the L1 norm L_o of the image gradient obtained in step D) and step E), and calculating the loss function value loss_o of the image;

[0012] G) Determine whether the loss function value loss_o and the number of iterations meet the threshold condition. If so, output I_c as the vignetting corrected image. If not, proceed to the next step.

[0013] H) using the gradient descent optimization method, calculating the optimization vector of the vignetting parameter through the difference between the loss function values ​​before and after image correction, obtaining the new parameter P_c, and increasing the number of iterations by 1;

[0014] I) Calculate a new vignetting model according to the optimized vignetting parameters, and replace the initial vignetting model V_0 with the new vignetting model, and repeat steps A) to I) until the threshold condition for exiting the loop is met.

[0015] The vignetting model used in step A) is a classic quartic polynomial model, as specifically shown in Formula 1, where x represents the distance between the pixel and the vignetting center, V(x) represents the vignetting degree at a distance x from the vignetting center, and a, b, and c are parameters.

[0016] V (x) =a+bx 2 +cx 4 (1)

[0017] Wherein, the image color space conversion method in step B) adopts the CIE standard;

[0018] The color intensity histogram division process in step C) is as follows: the three HSV spaces are divided into 20 parts (h1-h20, s1-s20, v1-v20. h, s, v represent the indexes of the histogram in the three spaces) in an equally spaced manner;

[0019] The color intensity histogram statistics process in step C) is as follows: traverse all pixels of the image, calculate which interval their h, s, and v fall in, and add 1 to the corresponding histogram element;

[0020] The information entropy statistical process in step D) is: extract all elements in the histogram with the same h and s indexes but different v indexes, and solve the information entropy of these elements. The information entropy calculation process is shown in equations (2) and (3). C_H represents the color intensity histogram calculated in step C), h j ,s k ,v iThey represent the histogram indexes of the histogram in step C) in the three directions of hue, saturation and intensity, and the range of j, k, i is 1-20. In formula (3), the upper and lower limits of ∑ represent that i changes from 1 to 20. In formula (2), the upper and lower limits of ∑ represent that j and k change from 1 to 20 respectively. The E_o finally solved is the color intensity entropy index.

[0021]

[0022] in:

[0023] Wherein, the method for solving the image gradient in step E) includes but is not limited to the Sobel operator, the Canny operator, the Laplace operator, etc.;

[0024] Among them, the vignetting correction loss function in step F) is as follows, E_o is the color intensity entropy index calculated in step D), L_o is the image gradient L1 norm calculated in step E), λ represents the regularization coefficient, and loss_o is the calculated vignetting correction loss function value.

[0025] loss_o=E_o+λ·L_o (4)

[0026] Wherein, the threshold condition in step G) is that the loss function value is less than 0.01 or the number of iterations is greater than or equal to the maximum number of iterations;

[0027] Among them, the gradient descent optimization method in step H) adopts the SGD technology.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention proposes a vignetting correction method that optimizes color intensity entropy and gradient sparsity. The method effectively utilizes the prior knowledge that vignetting will reduce the image color intensity entropy and increase the image gradient. The vignetting correction of the image is achieved by reasonably optimizing the color intensity entropy and gradient sparsity of the image, thereby avoiding the problems of large workload and poor reusability of traditional calibration-based vignetting methods. Compared with other vignetting correction methods, the method proposed in the present invention relies on more universal prior knowledge, has wider applicability, and can obtain more accurate vignetting correction results. In addition, the method improves the calculation speed through the optimization method of gradient descent, solving the technical problem that the single-image correction method is difficult to achieve real-time application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 1. is a schematic diagram of an algorithm flow of a vignetting correction method for optimizing color intensity entropy and gradient sparsity according to the present invention;

[0031] Figure 2 (A) to Figure 2(D) is a schematic diagram of converting an RGB image into an HSV space according to an embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of performing histogram statistics on an HSV spatial image according to an embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of solving the gradient of an HSV space image according to an embodiment of the present invention;

[0034] Figure 5 (A) to Figure 5 (D) is a schematic diagram of an iterative optimization process according to an embodiment of the present invention;

[0035] Figure 6 (A) to Figure 6 (H) is a schematic diagram of vignetting correction according to several embodiments of the present invention. DETAILED DESCRIPTION

[0036] In such Figure 1 In the specific embodiment of the present invention shown, the input image data is I_r, the output is the image I_c after vignetting correction, and the initial number of iterations iter=0 is set.

[0037] According to a vignetting correction method of color intensity entropy and gradient sparsity optimization in this embodiment, the method can achieve good results in image vignetting correction, such as Figure 1 As shown, the following steps are included:

[0038] A) The vignetting model used in this embodiment is a classic polynomial model as shown in equation (5), where x represents the distance between the pixel and the vignetting center of the image, the parameters are a, b, c, and V(x) represents the vignetting degree at a distance x from the vignetting center.

[0039] V (x) =a+bx 2 +cx 4 (5)

[0040] The image vignetting center position is calculated according to the image size, and the initial vignetting model V_0 is constructed according to the preset default parameters according to formula (5). Specifically, in this embodiment, the default parameters are a=0.5, b=0.2, c=0.1, and the distance x between the pixel and the vignetting center position is calculated by the Euclidean distance, as shown in formula (6). Where (i, j) is the horizontal and vertical coordinates of the pixel, i0, j0 represents the coordinates of the image vignetting center position, which is generally the center of the image, and can be calculated by dividing the horizontal and vertical dimensions of the input image by 2.

[0041]

[0042] B) Using the vignetting model V_0 in step A), the original image I_r is subjected to vignetting correction. The correction method is to divide the original image by the vignetting model, as shown in formula (7).

[0043] I_c=I_r / V_0 (7)

[0044] C) Use the CIE standard to convert the image I_c in step B) into color space and obtain the HSV space image I_h. The conversion equations are shown in equations (8)-(11), where equations (8)-(10) are the conversion formulas for H (hue), S (saturation) and V (intensity), respectively. Formula (11) is the intermediate variable calculation method required in the process. R, G, and B represent the values ​​of the pixels in the RGB image space, respectively. Max means finding the maximum value, and min means finding the minimum value. The result of converting the original image to the HSV space is shown in Figure 2 As shown, Figure 2 (A) represents the original input image, Figure 2 (B) represents the H channel component after HSV conversion. Figure 2 (C) represents the S channel component after HSV conversion. Figure 2 (D) represents the V channel component after HSV conversion. Through the above method, the color information (H and S) of the image can be separated from the intensity information (V), which is convenient for subsequent operations.

[0045]

[0046] V=Cmax(10)

[0047] in:

[0048] D) Divide the histogram groups in the HSV space for histogram statistics. This embodiment divides the histogram in an equally spaced manner, dividing the three HSV channels into 20 equal parts. The H value range is 0-360, the S value range is 0-1, and the V value range is 0-1. Therefore, in this embodiment, the group interval of H is 18, and the group intervals of S and V are both 0.05. The category labels of each group of histograms are respectively recorded as h1, h2, h3...h20, s1, s2, s3...s20, v1, v2, v3...v20. In this embodiment, the histogram statistics results have a total of 20*20*20=8000 groups.

[0049] E) Perform histogram statistics on the HSV domain image obtained in step C) according to the group interval divided in step D). The process is as follows: traverse all pixel points on the image, for each pixel point, extract its H, S, V value, and divide it by the group interval. The result is the index of each channel in the histogram, as shown in formula (12).

[0050]

[0051] For example, for a pixel with an HSV value of (165, 0.41, 0.77), its corresponding histogram is h9s8v15. Figure 3 The figure is a schematic diagram of an embodiment of performing histogram statistics on an HSV space image, wherein the horizontal direction represents the color and saturation direction, the vertical direction represents the intensity direction, and the grayscale value of each histogram area represents the number of pixels in the histogram. The larger the grayscale value, the more pixels there are in the histogram.

[0052] F) In HSV space, H and S represent the color information of the image, so the histograms with the same h and s groups are histograms with the same color. The histograms with the same V group are histograms with the same intensity. In this embodiment, for the histograms obtained in step E), histograms with the same color are extracted, for example: h1s1v1, h1s1v2, h1s1v3...h1s1v20, and the degree of aggregation of pixel intensity distribution in the above histograms with the same color is statistically analyzed. This embodiment uses information entropy to characterize this characteristic. The information entropy calculation process is shown in formula (13), where h j ,s k ,v i Respectively represent the histogram index in the three directions of hue, saturation and intensity. The range of j, k, i is 1-20. Indicates the color intensity histogram calculated in step C) at index h j s k v i The value of ∑ in formula (13) means that for i changing from 1 to 20, that is, for a fixed color and saturation histogram index, all intensity histogram indexes are traversed. The solution is That is the color intensity entropy under the color intensity index.

[0053]

[0054] G) Traverse all colors of the histogram, the traversal process is as follows: h1s1, h1s2, h1s3...h1s20, h2s1, h2s2...h2s20, h3s1...h20s1, h20s2...h20s20. For each color, calculate the information entropy of all histograms under each color according to step F). And the sum of the intensity information entropy under all colors is used as the entropy index of the whole image, that is, the color intensity entropy index E_o of the image I_h, as shown in formula (14), where the upper and lower limits of ∑ represent that j and k change from 1 to 20 respectively. .

[0055]

[0056] H) In this embodiment, the Laplacian operator is used to process the V component of the image I_h to calculate the gradient of the image. The Laplacian operator is shown in formula (15).

[0057]

[0058] The gradient information of the image is obtained as I_g, Figure 4 It is a schematic diagram of an embodiment of the present invention for solving the gradient.

[0059] I) Solve the L1 norm of the gradient image I_g obtained in step H), and the process is shown in formula (16) to obtain the evaluation index L_o for the image gradient information, where i, j represents the image pixel coordinates, and ∑ represents the sum of all pixels.

[0060]

[0061] J) Construct a vignetting optimization loss function as shown in formula (17), λ is a regularization coefficient, and in this embodiment, the regularization coefficient is 0.1. Use the gradient information evaluation index L_o obtained in step I) and the image entropy evaluation index E_o obtained in step G) to solve the loss function value loss of the image.

[0062] loss_o=E_o+λ·L_o (17)

[0063] K) Determine whether the loss obtained in step J) is less than a threshold value (set to 0.01 in this embodiment). If it is less than the threshold value, the last vignetting correction result I_c is output as the final result. If the loss function value does not reach the threshold value, the number of iterations iter is increased by 1, and it is determined whether the number of iterations reaches the maximum number (set to 100 in this embodiment). If the number of iterations reaches the maximum number, the loop is exited and the last vignetting correction result I_c is output as the final result.

[0064] L) If the judgment of the loss function value and the number of iterations in step K) both meet the exit condition, the vignetting parameters are optimized, and the vignetting correction effect is gradually optimized by an iterative method. In this embodiment, the gradient descent method is used to optimize the vignetting parameters, and the steps are to calculate the optimization vector of the vignetting parameters according to the ratio of the loss function change amount and the parameter change amount, and calculate the new vignetting parameters accordingly, as shown in equations (18)-(20).

[0065]

[0066] Where d represents the gradient operation, a, b, c are the vignetting parameters used in this iteration, and a', b', c' are the optimized vignetting parameters.

[0067] M) Inputting the new vignetting parameters a', b', c' calculated in step L) into step A), calculating a new vignetting model, and repeating steps A) to L) until the condition for iterative exit is met. Figure 5 (A) to (D) are schematic diagrams of an iterative optimization process according to an embodiment of the present invention, wherein: Figure 5 (A) represents the original image, Figure 5 (B) shows the correction result based on the default vignetting parameters. Figure 5 (C) shows the correction result after 3 iterations. Figure 5 (D) shows the final vignetting correction result when exiting the loop.

[0068] Figure 6 (A) to (H) are schematic diagrams of vignetting correction according to several embodiments of the present invention, wherein: Figure 6 (A) Figure 6 (C) Figure 6 (E) Figure 6 (G) is the original image, Figure 6 (B) Figure 6 (D) Figure 6 (F) Figure 6 (H) is the corresponding vignetting correction result.

[0069] Advantages of the present invention include:

[0070] (1) The present invention proposes a method for calculating the color intensity entropy of a statistical image, which can effectively characterize the brightness balance of different colors in the image by decomposing the color intensity of the image;

[0071] (2) The present invention proposes a priori constraint that vignetting will lead to an increase in the color intensity entropy of the image. Compared with the priori constraint that only relies on intensity information, the priori constraint proposed by the present invention has strong universality and is applicable to a wider range of scenarios;

[0072] (3) The present invention proposes a vignetting correction method that optimizes color intensity entropy and gradient sparsity, which can quickly and accurately implement image vignetting correction. This method does not require calibration and can be applied to various RGB images, which is of great significance for the practical application of vignetting correction algorithms.

Claims

1. A vignetting correction method based on color intensity entropy and gradient sparsity optimization, characterized in that The following steps are involved: A) establishing an initial vignetting model according to the image size and initial vignetting parameters, using the classic quartic polynomial model shown in formula (1) as the vignetting model, where x represents the distance between the pixel and the vignetting center of the image, the parameters are a, b, c, and V(x) represents the vignetting degree at a distance x from the vignetting center, and using the model to perform initial correction on the vignetting to obtain an initial vignetting corrected image; V (x) =a+bx 2 +cx 4 (1) B) converting the image in step A) into HSV space, and performing histogram statistics on the image using parameters including at least one of equal interval group spacing and non-equal interval group spacing; C) performing color intensity entropy statistics on the histogram of the HSV domain obtained in step B), and using it as the color intensity entropy evaluation index of the image; the color intensity entropy statistical process includes solving the entropy value of the intensity (V) distribution of the histogram with the same hue (H) and saturation (S), and the solution process is shown in equations (2) and (3), where Represents the color intensity histogram h j s k v i The value of h j ,s k ,v i They represent the histogram indexes of the step histogram in the three directions of hue, saturation and intensity respectively. max_j, max_k and max_i represent the maximum values ​​of the histogram index respectively. The final solved E_o is the color intensity entropy index. in: D) for the vignetting-corrected image obtained in step A), solving the gradient of at least one of the RGB image itself, the HSV image, and the V channel of the HSV image; E) calculating the norm of the image gradient obtained in step D), and using it as the gradient index evaluation of the image; F) Combining the image entropy evaluation index obtained in step C) with the image gradient evaluation index obtained in step E), as shown in formula (4), constructing an image vignetting optimization loss function; wherein E_o is the color intensity entropy index calculated in step C), L_o is the image gradient index evaluation calculated in step E), λ represents the regularization coefficient, loss_o is the calculated vignetting correction loss function value, loss_o=E_o+λ·L_o (4) G) optimizing the vignetting parameters using a convex function optimization method for the loss function value obtained in step F); H) determining whether the optimization termination condition is met, if the termination condition is met, outputting the last vignetting correction result, if the termination condition is not met, replacing the initial vignetting parameters with the optimized vignetting parameters, and looping steps A) to H) until the iteration is terminated.

2. The vignetting correction method of color intensity entropy and gradient sparsity optimization according to claim 1, characterized in that: The vignetting model includes at least one of a quartic polynomial model, a cos quartic model, and a Gaussian model.

3. The vignetting correction method of claim 1 with color intensity entropy and gradient sparsity optimization, characterized in that: The initial correction process is to divide the original image by the initial vignetting model, as shown in formula (5), where V_0 is the initial vignetting model, I_r is the original image, and I_c is the correction result. I_c=I_r / V_0 (5).

4. The vignetting correction method of color intensity entropy and gradient sparsity optimization according to claim 1, characterized in that: The entropy value includes at least one of information entropy, cross entropy, and joint entropy.

5. The vignetting correction method of claim 1 with color intensity entropy and gradient sparsity optimization, characterized in that: In the step D), at least one of the Soble operator, the Canny operator and the Laplace operator is used to solve the image gradient.

6. The vignetting correction method of color intensity entropy and gradient sparsity optimization according to claim 1, characterized in that: The operation of obtaining the norm in step E) uses at least one of the L0 norm, the L1 norm, and the L2 norm.

7. The vignetting correction method of claim 1 with color intensity entropy and gradient sparsity optimization, characterized in that: In the step F), the operation of constructing the vignetting optimization loss function adopts at least one of weighted addition, logarithmic addition, and exponential addition.

8. The vignetting correction method of color intensity entropy and gradient sparsity optimization according to claim 1, characterized in that: In the step G), the convex function optimization method includes at least one of the gradient descent method, the stochastic gradient descent method, and the dual descent method.

9. The vignetting correction method of claim 1 with color intensity entropy and gradient sparsity optimization, characterized in that: In the step H), the iteration termination condition adopted includes at least one of reaching the maximum number of iterations, the loss is lower than a threshold, and the loss no longer decreases.

Images