A semi-reference image defogging algorithm evaluation method

Abstract

The application discloses a semi-reference image defogging algorithm evaluation method and belongs to the technical field of image processing. The application adopts four evaluation methods to obtain evaluation scores of defogged images, including dark channel prior, local average and contrast normalization coefficient of natural images, gradient information of a contrast defogged image and an original fog image, and saturation information of the contrast defogged image and the original fog image, and then the four evaluation scores are weighted and summed to obtain a final evaluation score. The application combines a no-reference quality evaluation index and a reference quality evaluation index, the no-reference evaluation index is used to measure the defogging degree of an image, and the reference evaluation index measures the degree of color deviation and abnormal texture of an image by comparing the gradient and saturation between the original fog image and the defogged image. The possibility of each pixel point leading to color deviation and abnormal texture is quantitatively measured by the gradient and saturation change degree of the defogged image compared with the original fog image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a semi-reference image dehazing algorithm evaluation method. Background Technology

[0002] Images captured from outdoor scenes often suffer from varying degrees of degradation due to atmospheric particles and water droplets, and this degradation becomes even more severe in foggy scenes. Image dehazing algorithms aim to recover clear, fog-free images from input captured by a camera in foggy scenes, which is of great significance for high-level vision tasks such as object detection and semantic image segmentation.

[0003] Image dehazing algorithm evaluation methods aim to provide fair and subjective scores that reflect the dehazing results obtained by different algorithms. Evaluation methods are generally divided into two categories: reference-based and non-reference-based. Non-reference-based evaluation methods provide an objective assessment of an image without requiring any additional information; reference-based evaluation methods require additional information (such as gradient maps and depth maps) to evaluate the image. However, since it is difficult to readily obtain both fogged images and their strictly corresponding clear images in practical applications, most mainstream image dehazing algorithms are currently based on synthetic fog datasets and use reference-based evaluation metrics such as SSIM and PSNR. In recent years, several new evaluation methods have been proposed. The paper "LKChoi, J.You and ACBovik, 'Referenceless Prediction of Perceptual FogDensity and Perceptual Image Defogging,'" in IEEE Transactions on Image Processing, vol.24, no.11, pp.3888-3901, Nov.2015, doi:10.1109 / TIP.2015.2456502, discloses a referenceless evaluation metric called FADE. This metric calculates 12 sub-feature metrics for each specified-size block of the image to be evaluated, and then performs operations such as averaging to obtain the score for each image. This metric can effectively measure the performance of image defogging algorithms in real-world scenes; however, due to the large number of metrics used, the algorithm has high complexity. In the paper "Liu W, Zhou F, Lu T, Duan J, Qiu G. Image Defogging Quality Assessment: Real-World Database and Method. IEEE Trans Image Process. 2021; 30: 176-190. doi:10.1109 / TIP.2020.3033402.Epub 2020Nov 18.PMID:33119509", Liu W et al. proposed a quality assessment index, FRFSIM, based on their own dataset MRFID. This index can measure the defogging effect in real-world scenarios. However, FRFSIM requires both a clear image of the corresponding scene and a defogging image. In practical applications, it can only be used to evaluate the defogging effect of synthetic fog.Chinese patent application No. 202010364743.2, entitled "An Evaluation Method for Image Dehazing Algorithms," proposes a full-reference evaluation index based on the VSI algorithm, which acquires foggy and fog-free image pairs of the same scene in different cities at different time periods. This index can focus on the quality of image regions that are more likely to attract human attention based on visual saliency. However, it cannot be applied to general multi-scene, large-scale datasets. Chinese patent application No. 201810359895.6, entitled "A Referenceless Objective Evaluation Method for Dehazing Effect," proposes a referenceless image quality evaluation index. Starting from the characteristics of dehazed images, it comprehensively considers two aspects: image sharpness (based on the degree of contrast enhancement and fog retention) and color fidelity (based on the degree of hue shift and oversaturation). It can be effectively used for the comprehensive evaluation of dehazed image quality; however, this evaluation index will exhibit some distortion when dealing with color casts, local blocks, and other phenomena in the image. In a Chinese patent application with application number 201710072123.X and application title "A Method and Apparatus for Evaluating the Dehazing Effect of Haze Images," four feature indicators were proposed based on the measurement of residual fog, contrast, and related saturation using prior information from the dark channel. First, the values ​​of the four feature indicators of the image to be evaluated were obtained, and then weighted processing was performed to obtain the quality score of the dehazed image. However, this method has certain limitations, as it may mistakenly identify abnormal color areas in the dehazed image as areas with good dehazing effects. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problems that existing image dehazing algorithm evaluation metrics cannot give accurate scores for image dehazing effects, and that there are no effective evaluation metrics for real-world foggy images after dehazing.

[0005] This invention provides a method for evaluating a semi-reference image dehazing algorithm, the method comprising:

[0006] Step 1, input the dehazed image I dehaze and the corresponding original fog image I hazy ;

[0007] Step 2: Obtain evaluation scores for the dehazed image using four evaluation methods:

[0008] Based on the DCP (Dark Channel Prior) method, obtain the no-reference evaluation score S of the dehazed image. 11 Evaluation score S 11 Used to estimate fog residue;

[0009] The no-reference evaluation score S of the dehazed image is obtained based on the MSCN index (local mean and contrast normalization coefficient of natural image).12 Evaluation score S 12 Used to estimate the residual level of fog;

[0010] By comparing the gradient information of the dehazed image with that of the original hazy image, a reference evaluation score S is obtained for the dehazed image. 21 Evaluation score S 21 Used to estimate whether an image produces abnormal textures after dehazing.

[0011] By comparing the saturation information of the dehazed image with that of the original hazy image, a reference evaluation score S is obtained for the dehazed image. 22 Evaluation score S 22 Used to estimate whether a color cast occurs in a dehazed image.

[0012] Step 3, for the unreferenced evaluation score S 11 No reference score S 12 A reference evaluation score S 21 And a reference evaluation score S 22 Perform a weighted summation and output the image quality score of the dehazed image.

[0013] Preferably, in step 2, the no-reference evaluation score S of the dehazed image is obtained. 11 Specifically, it includes:

[0014] If image I dehaze If the color space is not RGB, then it is converted to RGB color space, denoted as image I. If image I... dehaze If the color space is RGB, then image I is image I. dehaze .

[0015] The image I is divided into multiple non-overlapping image blocks P of size p×p;

[0016] In this process, the image can be uniformly divided into blocks in two ways:

[0017] Method 1: First, crop the image I. The size of the cropped image is an integer multiple of p in both width and height. Then, divide the cropped image into uniform and non-overlapping blocks to obtain multiple p×p image blocks P.

[0018] Method 2: Directly divide image I into uniform and non-overlapping blocks of size p×p. Usually, the division is performed from left to right or from top to bottom, or vice versa. After the division is completed, discard any blocks that are less than p×p.

[0019] For any image patch P, first calculate the minimum value of each pixel in P in the three color channels R (Red), G (Green), and B (Blue), to obtain R.min G min B min .

[0020]

[0021] Where P(x) represents a single pixel within P, and C R (P(x)), C G (P(x), C) B (P(x) represents the values ​​of the R, G, and B channels corresponding to pixel point P(x), respectively.

[0022] Find R min G min B min The minimum value in the score is taken as the score s of the image patch P:

[0023] s = P min =min(R) min G min B min (2)

[0024] Based on the normalization of s corresponding to all image patches P, and then the mean value, we obtain S. 11 :

[0025]

[0026] Where i represents the number of image patch P, s i S represents the score of image patch P with number i, N represents the number of image patches P, and s min ,s max Let S and S represent the minimum and maximum scores s corresponding to N image blocks P, respectively.

[0027] Preferably, in step 2, the no-reference evaluation score S of the dehazed image is obtained. 12 include:

[0028] Obtain the grayscale image I of image I gray ;

[0029] The preferred grayscale image conversion process is as follows:

[0030] I gray =0.3I R +0.59I G +0.11I B (4)

[0031] In formula (4), I R ,I G ,I B These represent the R, G, and B channel values ​​of image I, respectively;

[0032] Then, for the grayscale image I gray The boundaries are filled, and then a corresponding Gaussian-like filter kernel ω is selected for filtering to obtain a first response map μ with the same size as image I:

[0033]

[0034] Where k and l are the spatial indices of the current point, ω k,l This represents the value at the specified spatial index position in ω, where i and j represent the spatial indices of pixels in image I;

[0035] Obtain a second response map σ with the same size as image I:

[0036]

[0037] Calculate the no-reference rating score S 12 :

[0038]

[0039] Where W and H represent the width and height of image I, respectively.

[0040] Preferably, in step 2, the dehazed image has a reference evaluation score S. 21 Specifically, it includes:

[0041] Get Image I dehaze and I hazy Gradient value of each pixel:

[0042]

[0043] in, Let f(i) represent the second-order partial derivatives in the x and y directions, respectively, and let f(i) represent the image I. dehaze The value of the i-th pixel in the image, f(j) represents the image I. hazy The value of the j-th pixel in G. dehaze (i),G hazy (j) respectively represent I dehaze The gradient magnitude and I of the i-th pixel hazy The gradient magnitude of the j-th pixel;

[0044] Based on all G dehaze (i) Obtain the gradient matrix G dehaze Based on all G hazy (j) Obtain the gradient matrix G hazy For the gradient matrix G dehaze and G hazy Perform transformation and comparison processing to obtain the comparison metric value at the corresponding position, denoted as Num.i Where i = 2, 3, ..., W × H;

[0045] Statistics Num i The average value of S is obtained 21 :

[0046]

[0047] Among them, for the gradient matrix G dehaze and G hazy The transformation and comparison processes are performed to obtain the comparison metric values ​​as follows:

[0048] The gradient matrix G of size W×H dehaze and G hazy Flattened into a one-dimensional matrix M of size 1×(W×H) dehaze and M hazy Matrix M is concatenated along its dimensions. dehaze and M hazy This yields a matrix M of size 2×(W×H);

[0049] Sort the elements of the first and second rows of matrix M in descending or ascending order based on the size of the elements in the second row to obtain matrix M′; sort the elements of the second row of matrix M′ in descending or ascending order, and then extract the first row of matrix M′ to obtain a 1×(W×H) matrix M. compare ;

[0050] For matrix M compare Let the i-th element (i∈{2,3,........,W×H}) be compared with the preceding i-1 elements in turn (a total of i-1 comparisons), and let Num be the number of values ​​among the preceding i-1 elements that are less than or greater than the i-th element. i .

[0051] This invention calculates S 21 At the same time, it combines the gradient information of the original fog image to provide a reference evaluation index. It does not rely on the general features obtained from a large number of clear fog-free images and can effectively determine whether abnormal textures appear in the defogging image.

[0052] Preferably, in step 2, a reference evaluation score S is obtained for the dehazed image. 22 Specifically, it includes:

[0053] If I dehaze and I hazy If the color space is not HSV, then a color space conversion is performed to extract the S channel value of the HSV color space: I dehaze_s and I hazy_s ;if I dehaze and Ihazy If the color space is HSV, then the values ​​of the S channel are directly extracted to obtain image I. dehaze_s and Image I hazy_s ;

[0054] Obtain image I respectively dehaze_s and I hazy_s The gradient value of each pixel is used to obtain the gradient matrix G. dehaze_s and G hazy_s ;

[0055] For the gradient matrix G dehaze_s and G hazy_s Perform transformation and comparison processing to obtain the comparison metric (specific method is the same as Num). i (Same), denoted as Num_s i , i = 2, 3, ..., W × H;

[0056] Statistics Num_s i The average value of S is obtained 22 :

[0057]

[0058] Preferably, if I dehaze and I hazy If the color space is RGB, the value of the S channel can be obtained in the following way:

[0059]

[0060]

[0061] Where x represents the pixel position of the image, R(x), G(x) and B(x) represent the pixel values ​​of the image in the R, G and B color channels, respectively, and S(x) represents the pixel value of the S channel.

[0062] In this invention, gradient information from the original fog image is incorporated to provide a reference evaluation index. It does not rely on general features obtained from a large number of clear, fog-free images and can effectively determine whether color cast has occurred in the defogging image.

[0063] The technical solution provided by this invention brings at least the following beneficial effects:

[0064] (1) This invention combines a no-reference quality evaluation index and a reference quality evaluation index. The no-reference evaluation index is used to measure the degree of dehazing of the image, while the reference evaluation index evaluates the degree of color cast and abnormal texture of the image by comparing the gradient and saturation between the original hazy image and the dehazed image.

[0065] (2) The probability of each pixel causing color cast and abnormal texture is quantitatively measured by the degree of gradient and saturation change of each pixel in the dehazed image compared with the original hazy image. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart of a semi-reference image dehazing algorithm evaluation method provided in an embodiment of the present invention.

[0068] Figure 2 In this embodiment of the invention, M compare A schematic diagram of the matrix construction process. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0070] This invention provides a semi-referenced image dehazing algorithm evaluation method. This method can be used to evaluate the output image of a hazy image after processing by different image dehazing algorithms. By combining information from the input hazy image corresponding to the output image, a score consistent with human subjective evaluation is given to the output image. This invention uses a combination of referenced and non-referenced evaluation metrics to provide reliable evaluation scores for the output images of real-world hazy images processed by different image dehazing algorithms. The method considers two aspects: the degree of fog residue in the image and whether the dehazed image produces color cast or abnormal textures. For the degree of fog residue in the output image, two non-referenced metrics, dark channel prior and MSCN, are used for estimation. For the presence of color cast and abnormal textures in the output image, two referenced metrics, gradient and saturation, are used in conjunction with the original hazy image for judgment.

[0071] like Figure 1 As shown, the specific implementation steps of a semi-reference image dehazing algorithm evaluation method provided in this embodiment of the invention include:

[0072] Step S1, input the dehazed image I dehaze and the corresponding original fog image I hazy ;

[0073] Step S2: Obtain the no-reference evaluation score S of the dehazed image according to the DCP method. 11 :

[0074] Step S201, for the output image I after processing by different image dehazing algorithms dehaze (In this embodiment, image I) dehaze The color space is RGB), and it is denoted as image I. The width and height of image I are W and H, respectively. I is divided into multiple non-overlapping image blocks P of size 10×10 in the horizontal and vertical directions, resulting in N image blocks P. After the division, blocks smaller than 10×10 are discarded.

[0075] For a given image patch P, first calculate the minimum value R0 of each pixel in P across the R, G, and B color channels. min G min B min Calculate according to formula (1);

[0076] Step S202, obtain R min G min B min The minimum value in the score is taken as the score s of the image patch P.

[0077] Step S203: Normalize the values ​​of s corresponding to the N image blocks P to 0-100, and then calculate the mean to obtain S. 11 The specific calculation formula is shown in formula (3).

[0078] Step S3: Obtain the no-reference evaluation score S of the dehazed image based on the MSCN index. 12 :

[0079] Step S301: Obtain the grayscale image I of image I according to formula (4). gray ;

[0080] Step S302, for grayscale image I gray The boundaries are filled; in this embodiment, the fill value is set to 3.

[0081] Step S303: Select a Gaussian-like filter kernel ω, as shown below, and use ω to filter the grayscale image I. gray Perform filtering with a step size of 1;

[0082]

[0083] According to formula (5), the response map μ with the same size as image I is obtained. In this embodiment, the values ​​of K and L are both 3.

[0084] Step S304, similar to step S303, obtains the response map σ with the same size as image I according to formula (6);

[0085] Step S305, calculate S according to formula (7) 12 .

[0086] Step S4: By comparing the gradient information of the dehazed image with that of the original hazy image, a reference evaluation score S is obtained for the dehazed image. 21 :

[0087] Step S401, calculate image I according to formula (9) respectively. dehaze and Image I hazy The gradient value of each pixel in G: dehaze (i),G hazy (j); based on all G dehaze (i) Obtain the gradient matrix G of size W×H. dehaze Based on all G hazy (j) Obtain the gradient matrix G of size W×H. hazy ;

[0088] Step S402, as follows Figure 2 As shown, the gradient matrix G of size W×H is... dehaze and G hazy Flattened into a one-dimensional matrix M of size 1×(W×H) dehaze and M hazy Then M dehaze and M hazy The concatenation yields a new matrix M of size 2×(W×H). The elements of the first and second rows of M are then simultaneously sorted in descending (or ascending) order based on the size of the elements in the second row, resulting in matrix M′. The elements of the second row of M′ are then sorted in descending (or ascending) order. At this point, the elements in the same column of the first and second rows represent the same pixel G. dehaze (i),G hazy The value of (i) is then taken, and the first row of M′ is extracted to obtain a 1×(W×H) matrix M. compare ;

[0089] Then, from matrix M compare Starting with the second element, compare the i-th element (i∈{2,3,........,W×H}) with the preceding i-1 elements (compare i-1 times). Record the number of values ​​in the preceding i-1 elements that are smaller than the i-th element as Num. i .

[0090] Step S403, based on the Num obtained from the comparison i According to formula (9), S is obtained. 21 .

[0091] Step S5: By comparing the saturation information of the dehazed image and the original hazy image, a reference evaluation score S is obtained for the dehazed image. 22 :

[0092] Step S501, according to formulas (11) and (12), I dehaze and I hazy Convert to HSV space and obtain the value of the S channel to get I. dehaze_s and I hazy_s ;

[0093] Step S502, calculate I according to formula (9) respectively dehaze_s and I hazy_s The gradient value of each pixel in the image is based on I. dehaze_s Each gradient value yields a gradient matrix G of size W×H. dehaze_s Based on I hazy_s Each gradient value yields a gradient matrix G of size W×H. hazy_s ;

[0094] Step S503: Following the same processing method as step S402, process the gradient matrix G. dehaze_s and G hazy_s Perform transformation and comparison processing to obtain the comparison metric value, denoted as Num_s. i , i = 2, 3, ..., W × H;

[0095] Step S503, based on the Num_s obtained from the comparison i According to formula (10), S is obtained. 22 .

[0096] Step S6: Calculate the final score using a weighted sum.

[0097] Score = α1S 11 +α2S 12 +α3S 21 +α4S 22 (14)

[0098] Wherein, α1, α2, α3, and α4 represent the weighting coefficients of the four evaluation scores. In this embodiment, their values ​​are 0.2, 0.3, 0.25, and 0.25, respectively.

[0099] The above description is merely a detailed illustration of specific embodiments of the present invention and is not intended to limit the invention. Various substitutions, modifications, and improvements made by those skilled in the art without departing from the principles and scope of the present invention should be included within the protection scope of the present invention.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0101] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A method for evaluating a semi-reference image dehazing algorithm, characterized in that, Includes the following steps: Step 1, input the dehazed image. and the corresponding original fog image ; Step 2: Obtain evaluation scores for the dehazed image using four evaluation methods: Based on the dark channel prior method, obtain the no-reference evaluation score of the dehazed image. ; The no-reference evaluation score of the dehazed image is obtained based on the local average and contrast normalization coefficient of the natural image. ; By comparing the gradient information of the dehazed image with that of the original hazy image, a reference evaluation score is obtained for the dehazed image. ; By comparing the saturation information of the dehazed image and the original hazy image, a reference evaluation score is obtained for the dehazed image. ; Step 3, for scores without reference evaluation No reference score Reference evaluation scores are available. And reference evaluation scores Perform weighted summation and output the image. Image quality evaluation score; In step 2, a reference evaluation score is obtained for the dehazed image. Specifically, it includes: Get Image and The gradient value of each pixel is used to obtain the gradient matrix. and ; For the gradient matrix and Perform transformation and comparison processing to obtain the comparison metric value at the corresponding position, denoted as . ,in, , Representing an image Width and height; statistics The average value is obtained : ; Specifically, the transformation and comparison of the two gradient matrices to obtain the comparison metric are performed as follows: Flatten the two gradient matrices into one-dimensional matrices, denoted as . and , where the matrix Corresponding image , Corresponding image ; And concatenate the matrix according to its dimensions. and , thus obtaining a two-dimensional matrix ; matrix The elements in the first and second rows of the matrix are simultaneously sorted in descending or ascending order based on the size of the elements in the second row to obtain the matrix. ; to matrix The elements in the second row are arranged in descending or ascending order, then the matrix is... Extracting the first row yields a 1×(W×H) matrix. ; For matrix The first in The nth element is compared sequentially. Each element and the preceding The size of each element, and the preceding... Among the elements, the one less than or greater than the first one The number of values ​​for each element is denoted as . ,in .

2. The method as described in claim 1, characterized in that, In step 2, obtain the no-reference evaluation score of the dehazed image. Specifically, it includes: If the image If the color space is not RGB, then perform a color space conversion to RGB and denote it as image I; if the image If it's in RGB color space, then directly use the image... As image I; Image I is divided into uniform and non-overlapping blocks to obtain multiple blocks of size I. Image Patch ; Calculate each image patch Score s: Calculate the score for each image patch. The minimum value of each pixel in the R, G, and B color channels is obtained. ;Will The minimum value in the image is used as the current image patch. The score s; For all image patches of image I The score s is normalized to obtain the normalized score. Then based on all image patches Normalized score The mean value is given as a no-reference evaluation score. .

3. The method as described in claim 2, characterized in that, For all image patches The score s is normalized as follows: ； in, These represent all image blocks of image I. The minimum and maximum values ​​of the score s.

4. The method as described in claim 1, characterized in that, In step 2, obtain the no-reference evaluation score of the dehazed image. include: If the image If the color space is not RGB, then perform a color space conversion to RGB and denote it as image I; if the image If it's in RGB color space, then directly use the image... As image I; Obtain the grayscale image of image I ; For grayscale images Fill the boundaries, and then select the appropriate Gaussian-like filter kernel. The first response image is obtained by performing filtering. : ； in, Represents the first response diagram The arbitrary pixel value, where i and j represent the spatial index of the pixel in the image, and k and l represent the Gaussian-like filter kernel. Spatial index location, express The values ​​at the specified spatial index positions, K and L represent respectively. Width and height; Calculate the second response map : ； Calculate the score without reference : ； Where W and H represent the width and height of image I, respectively.

5. The method as described in claim 4, characterized in that, Obtain the grayscale image of image I for: ； in, These represent the R, G, and B channel values ​​of image I, respectively.

6. The method as described in claim 1, characterized in that, In step 2, a reference evaluation score is obtained for the dehazed image. Specifically, it includes: like and If the color space is not HSV, then a color space conversion is performed to extract the S channel value of the HSV color space: and ;like and If the color space is HSV, then the S channel value is directly extracted to obtain the image. and images ; Acquire images separately and The gradient value of each pixel is used to obtain the gradient matrix. and ; For the gradient matrix and Perform transformation and comparison processing to obtain the comparison metric value, denoted as... , ; statistics The average value is obtained : .

7. The method as described in claim 6, characterized in that, like and If the color space is RGB, then the value of the S channel extracted from the HSV color space is: ； ； in, Represents the pixel position of an image. , and These represent the pixel values ​​in the R, G, and B color channels of the image, respectively. This represents the pixel value of the S channel.

Images