Aerial image defogging method for enhancing details by using transmissivity

Abstract

The invention discloses an aerial image defogging method for enhancing details by using transmissivity. The aerial image defogging method consists of three parts of solving an atmospheric scattering model, extracting image details and fusing the atmospheric scattering model and the image details. Specifically, an atmospheric light value is obtained through a proposed region search scoring method, and meanwhile, the transmissivity obtained from a smoother and filtered fog image is substituted into an atmospheric scattering model to solve a defogged image. In the image detail processing, the smoothed image is subjected to weight smoothing, sharpening and filtering operation, the transmissivity is introduced into the regional variance weight in the filtering operation, a filter is controlled by the transmissivity to perform smoothing or sharpening operation, and the filtered image is subtracted from the smoothed image to obtain a detail image. And finally, a defogged image is formed by linearly fusing an atmospheric scattering model inverse solution image and an image after detail processing. According to the method, the details of the original image after defogging are effectively reserved, and the definition of the restored image is improved.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to an aerial image defogging method utilizing transmittance to enhance details. Background Art

[0002] Aerial images are widely used in the field of air-to-ground target detection, but the quality of images collected in foggy weather is severely reduced, which has a serious impact on subsequent image applications such as target recognition. Therefore, image defogging has important research value in improving the quality of images collected in foggy days. At present, the main methods for defogging single images are: image defogging methods based on atmospheric scattering models, methods based on image enhancement, and image defogging based on deep learning. When the existing methods perform image defogging, there are problems such as improper solution of atmospheric light values ​​causing halos and artifacts in the defogged image, and loss of image details caused by the inverse solution of the atmospheric scattering model. In view of the problems of poor defogging effect and loss of details in general image defogging algorithms under large scenes of aerial images, the present invention introduces transmittance in the solution of regional variance, effectively retains the near-far characteristics in the image details, and improves the quality of the defogged image.

[0003] To deal with the haze problem in images, HE et al. found that in most non-sky local areas, some pixels in at least one color channel have very low intensity, and proposed single image dehazing with dark channel prior (He K, Sun J, Tang X. Single image haze removal using dark channel prior [C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009: 1956-1963). KANTI et al. divided the image into images with and without color cast, and performed color balance through nonlinear transformation, and then made fine adjustments based on color cast to achieve image dehazing (Kanti Dhara S, Roy M, Sen D, et al.. Colorcast dependent image dehazing via adaptive airlight refinement and non-linear color balancing [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31 (5): 2076-2081). Summary of the invention

[0004] The present invention provides an aerial image defogging method for enhancing details by utilizing transmittance, comprising the following steps:

[0005] Step 1: Establishment of atmospheric scattering model,

[0006] The expression of the atmospheric scattering model for images collected in a foggy environment is:

[0007] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0008] Where x represents a pixel, I represents a foggy image, J represents a fog-free image, that is, the defogging image, t(x) represents the transmittance, and A represents the atmospheric illumination value;

[0009] Step 2: Calculate the atmospheric light value and transmittance.

[0010] The atmospheric illumination value is obtained by the regional scoring method. The specific steps are as follows: First, adjust the size of the light value search box according to the size of the foggy image, and use the quadtree method to obtain the atmospheric light value. Generally, it is iterated 5 to 6 times and then stopped. That is, the size of the atmospheric light area is 1 / 4 of the entire image. 6 Therefore, the atmospheric light value search box size boxsize is determined as:

[0011]

[0012] Where H and L are the height and width of the image, and then the search speed of the region box is set. Finally, the contrast and brightness comprehensive score of the region pane is constructed to score the area covered by the search box;

[0013] When choosing contrast, choose the Weber contrast based on the human visual system, Weber contrast C Weber is the normalized difference between the object value and the background value, and the formula is as follows:

[0014]

[0015] Among them, C Weber is the Weber contrast of the target point, I c.object is the pixel value of the target point in the image, I c.ground is the average value of pixels within the target point area;

[0016] The atmospheric illumination regional score needs to consider the regional contrast and regional brightness value. Therefore, the regional maximum value and regional average value are used as the brightness score. The regional score formula is as follows:

[0017]

[0018] Among them, c∈{r,g,b} is the red, green and blue channels of the image, B is the search box area, and finally, in the area with the highest score Ω, the distance between the color vector of each pixel and the maximum value of RGB is calculated, and the pixel with the smallest distance is selected as the estimated value of the global atmospheric illumination value, that is:

[0019]

[0020] Among them, x r is the value of pixel x in the red channel, x g is the value of pixel x in the green channel, x b is the value of pixel x in the blue channel, and the formula for solving the transmittance is:

[0021]

[0022] in A control factor δ = 0.6 is introduced in the formula to constrain the rate of change;

[0023] According to the atmospheric light value A(x) and transmittance t(x) obtained by equations (5) and (6), the atmospheric scattering model formula (1) is inversely solved, that is, the dehazed image solved by the atmospheric scattering model is:

[0024]

[0025] Step 3: Introduce the regional variance of transmittance,

[0026] In the smoothing and sharpening filtering technology, local variance is introduced as the edge perception weight to form a weighted smoothing and sharpening filter to improve the detail enhancement effect of the fog image. The local area variance weight Γ G (x) The formula is as follows:

[0027]

[0028] in N is the number of pixels in the local region ξ, and x' is the number of pixels in the region ξ;

[0029] The transmittance factor γ=e is introduced into the regional variance weight t(x) , to meet the detail extraction requirements of foggy images, the regional variance weight after the transmittance factor is introduced The definition is as follows:

[0030]

[0031] Where 1≤γ≤e, if x is located in the edge area, then Γ G (x) is usually greater than 1, then γ will have an effect on Γ G(x) is enhanced to a greater extent; if x is in the smooth region, then Γ G (x) is usually less than 1, then γ will have an effect on Γ G (x) weaken to a certain extent,

[0032] The cost function E used by the weighted smoothing and sharpening filter is:

[0033]

[0034] in 1 / θ 2 ∈[0.01,100], η=κ'ε, solve We can get:

[0035]

[0036] Step 4: Introduce transmittance detail extraction,

[0037] Because the fog phenomenon can distinguish the foreground and background of the image, detail enhancement of the entire foggy image alone will cause image distortion. By taking advantage of the weighted smoothing and sharpening filter, which can both smooth and sharpen, the foreground area of ​​the foggy image is smoothed and the foreground area is sharpened. Definition in is the transmittance threshold set. By statistically analyzing the transmittance distribution of a large number of foggy images, the median transmittance of most foggy images is 0.7-0.8. If is too small, the value of κ' in the foreground will be too large, resulting in excessive image sharpening. When it is too large, the background part of the image will have a too small value of κ', resulting in a smooth transition of the image. Therefore, the value of κ' is:

[0038]

[0039] When the transmittance is greater than the threshold, κ'>1, the filter is adjusted to a sharpening filter; when the transmittance is less than the threshold, κ'<1, the filter is adjusted to a smoothing filter. The final detail map is defined as follows:

[0040]

[0041] Where D(p) is the detail map of region ξ, and the weight ω k for:

[0042]

[0043] in, is a regularization factor to ensure s is a predefined scale parameter, and σ 2is the average of the regional variance of the entire image;

[0044] Step 5: Linear fusion image restoration,

[0045] The final dehazed image is obtained by linearly merging the image solved by the atmospheric scattering model with the detail sharpened image, namely:

[0046] R(x)=J(x)+D(x) (15)

[0047] Where R(x) is the final dehazed image, J(x) is the image solved by formula (7), and D(x) is the detail image extracted in step 2.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The method described in the present invention aims at the problem that it is difficult to accurately estimate the atmospheric light value of aerial images, and proposes a regional scoring method to obtain the optimal atmospheric light value; at the same time, the current defogging algorithm has the problem of detail loss when processing aerial foggy images, and proposes a weighted smoothing-sharpening filtering algorithm that introduces transmittance into the regional variance to sharpen the details of the defogged image, thereby improving the clarity of the defogged image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for describing the specific implementation or the prior art will be briefly introduced below. Obviously, the drawings described below are only one implementation of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 : This is a comparison of images before and after processing by an embodiment of the present invention: wherein a is a natural foggy image, b is a defogging image using the dark channel prior method; c is a defogging image using nonlinear color balance; and d is a defogging image using the method of the present invention. DETAILED DESCRIPTION

[0052] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited to the contents described.

[0053] Step 1: Establishment of atmospheric scattering model,

[0054] Images collected in foggy environments, such as Figure 1 As shown in a, the expression of the atmospheric scattering model is:

[0055] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0056] Where x represents a pixel, I represents a foggy image, J represents a fog-free image, that is, the defogging image, t(x) represents the transmittance, and A represents the atmospheric illumination value;

[0057] Step 2: Calculate the atmospheric light value and transmittance.

[0058] The atmospheric illumination value is obtained by the regional scoring method. The specific steps are as follows: First, adjust the size of the light value search box according to the size of the foggy image, and use the quadtree method to obtain the atmospheric light value. Generally, it is iterated 5 to 6 times and then stopped. That is, the size of the atmospheric light area is 1 / 4 of the entire image. 6 Therefore, the atmospheric light value search box size boxsize is determined as:

[0059]

[0060] Where H = 600 and L = 600 are the height and width of the image, and then the search speed of the region box is set to 4. Finally, the contrast and brightness comprehensive scores of the region pane are constructed to score the area covered by the search box;

[0061] When choosing contrast, choose the Weber contrast based on the human visual system, Weber contrast C Weber is the normalized difference between the object value and the background value, and the formula is as follows:

[0062]

[0063] Among them, C Weber is the Weber contrast of the target point, I c.object is the pixel value of the target point in the image, I c.ground is the average value of pixels within the target point area;

[0064] The atmospheric illumination regional score needs to consider the regional contrast and regional brightness value. Therefore, the regional maximum value and regional average value are used as the brightness score. The regional score formula is as follows:

[0065]

[0066] Among them, c∈{r,g,b} is the red, green and blue channels of the image, and B is the search box area; finally, in the area with the highest score Ω, the distance between the color vector of each pixel and the maximum value of RGB is calculated, and the pixel with the smallest distance is selected as the estimated value of the global atmospheric illumination value, that is:

[0067]

[0068] Among them, x r is the value of pixel x in the red channel, x gis the value of pixel x in the green channel, x b is the value of pixel x in the blue channel; the formula for solving the transmittance is:

[0069]

[0070] in A control factor δ = 0.6 is introduced in the formula to constrain the rate of change;

[0071] According to equations (5) and (6), the atmospheric light value A(x) and transmittance t(x) can be obtained by inversely solving the atmospheric scattering model formula (1), that is, the cloud image solved by the atmospheric scattering model is:

[0072]

[0073] Step 3: Introduce the regional variance of transmittance,

[0074] In the smoothing and sharpening filtering technology, local variance is introduced as the edge perception weight to form a weighted smoothing and sharpening filter to improve the detail enhancement effect of the fog image. The local area variance weight Γ G (x) The formula is as follows:

[0075]

[0076] in N is the number of pixels in the local area ξ. According to the boxsize calculated by formula (17), N=81, and x' is the pixel in the area ξ;

[0077] The transmittance factor γ=e is introduced into the regional variance weight t(x) , to meet the detail extraction requirements of foggy images; the regional variance weight after introducing the transmittance factor is defined as follows:

[0078]

[0079] Where 1≤γ≤e, if x is located in the edge area, Γ(x) is usually greater than 1, and γ will enhance Γ(x) to a large extent; if x is located in the smooth area, Γ(x) is usually less than 1, and γ will weaken Γ(x) to a certain extent; the cost function E used by the final weighted smoothing and sharpening filter is,

[0080]

[0081] in 1 / θ 2 =4,η=κ'ε, solve Available

[0082]

[0083] Step 4: Introduce transmittance detail extraction,

[0084] Because the fog phenomenon can distinguish the foreground and background of the image, detail enhancement of the entire foggy image alone will cause image distortion. Taking advantage of the weighted smoothing and sharpening filter, the foreground area of ​​the foggy image is smoothed and the foreground area is sharpened. Definition in is the set transmittance threshold; by statistically analyzing the transmittance distribution of a large number of foggy images, the median transmittance of most foggy images is 0.7-0.8. If is too small, the value of κ' in the foreground will be too large, resulting in excessive image sharpening. If it is too large, the background part of the image will have a too small value of κ', resulting in a smooth transition of the image. The value of κ' is

[0085]

[0086] When the transmittance is greater than 0.75, κ'>1, the filter is adjusted to sharpening filtering; when the transmittance is less than 0.75, κ'<1, the filter is adjusted to smoothing filtering; the final detail image D(p) is defined as follows:

[0087]

[0088] Where D(p) is the detail map of region ξ, and the weight ω k for:

[0089]

[0090] in, is a regularization factor to ensure s is a predefined scale parameter set to 1, and σ 2 is the average of the regional variance of the entire image;

[0091] Step 5: Linear fusion image restoration,

[0092] The final dehazed image is obtained by linearly merging the image solved by the atmospheric scattering model with the detail sharpened image, namely:

[0093] R(x)=J(x)+D(x) (15)

[0094] Where R(x) is the final dehazed image, such as Figure 1 As shown in (d), J(x) is the image obtained by solving formula (7), and D(x) is the detail image extracted in step 2.

[0095] In order to further illustrate the advantages of the method of the present invention, Figure 1 The dehazed image obtained by using the dark channel prior method described in the background technology is as follows: Figure 1 As shown in b, the defogging image obtained by using the nonlinear color balance defogging method in the background technology is as follows Figure 1 As shown in c, Figure 1 It can be seen that the clarity of the photos after defogging using each method is significantly improved. Among them, the clarity and effect of the photos processed by the nonlinear color balance defogging method are slightly better than those of the dark channel prior method, and the clarity and effect of the photos processed by the method of the present invention are significantly better than those of the dark channel prior method and the nonlinear color balance defogging method.

[0096] The above describes the main technical features and basic principles of the present invention and the related advantages. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the concept or basic features of the present invention. Therefore, no matter from which point of view, the above specific embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention.

[0097] In addition, it should be understood that although the present specification is described according to various implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for defogging aerial images by using transmittance to enhance details, characterized in that: The following steps are involved: Step 1: Establishment of atmospheric scattering model The expression of the atmospheric scattering model for collecting images in a foggy environment is: I(x)=J(x)t(x)+A(1-t(x)) (1) Where x represents a pixel, I represents a foggy image, J represents a fog-free image, that is, the defogging image, t(x) represents the transmittance, and A represents the atmospheric illumination value; Step 2: Calculation of atmospheric light value and transmittance The atmospheric illumination value is obtained by the regional scoring method. The specific steps are as follows: First, adjust the size of the light value search box according to the size of the foggy image. The atmospheric light value is obtained by the quadtree method. Generally, it is iterated 5 to 6 times and then stopped. That is, the size of the atmospheric light area is 1 / 4 of the entire image. 6 Therefore, the atmospheric light value search box size boxsize is determined as: Where H and L are the height and width of the image, and then the search speed of the region box is set. Finally, the contrast and brightness comprehensive score of the region pane is constructed to score the area covered by the search box; Among them, c∈{r,g,b} is the red, green and blue channels of the image, B is the search box area, and finally, in the area with the highest score Ω, the distance between the color vector of each pixel and the maximum value of RGB is calculated, and the pixel with the smallest distance is selected as the estimated value of the global atmospheric illumination value, that is: Among them, x r is the value of pixel x in the red channel, x g is the value of pixel x in the green channel, x b is the value of pixel x in the blue channel, and the formula for solving the transmittance is: in A control factor δ = 0.6 is introduced in the formula to constrain the rate of change; According to the atmospheric light value A(x) and transmittance t(x) obtained by equations (5) and (6), the atmospheric scattering model formula (1) is inversely solved, that is, the dehazed image solved by the atmospheric scattering model is: Step 3: Introduce the regional variance of transmittance In the smoothing and sharpening filtering technology, local variance is introduced as the edge perception weight to form a weighted smoothing and sharpening filter to improve the detail enhancement effect of the fog image. The local area variance weight Γ G (x) The formula is as follows: in N is the number of pixels in the local region ξ, and x' is the number of pixels in the region ξ; Step 4: Introducing transmittance detail extraction definition in The transmittance threshold is set, and the median transmittance of the foggy image obtained by statistical method is 0.7-0.8, and the value of κ' is: When the transmittance is greater than the threshold, κ'>1, the filter is adjusted to a sharpening filter; when the transmittance is less than the threshold, κ'<1, the filter is adjusted to a smoothing filter. The final detail map is defined as follows: Where D(p) is the detailed image of region ξ, ω k is the weight; Step 5: Linear fusion image restoration The final dehazed image is obtained by linearly merging the image solved by the atmospheric scattering model with the detail sharpened image, namely: R(x)=J(x)+D(x) (15) Where R(x) is the final dehazed image, J(x) is the image solved by formula (7), and D(x) is the detail image extracted in step 2.

2. The method for defogging an aerial image by enhancing details using transmittance according to claim 1, characterized in that: In step 2: In the selection of contrast, the Weber contrast based on the human visual system is used, and the formula is as follows: Among them, C Weber is the Weber contrast of the target point, I c.object is the pixel value of the target point in the image, I c.ground is the average value of pixels within the target point area; The regional maximum value and regional average value are used as the regional brightness score. The regional brightness score formula is as follows:

3. The method for defogging an aerial image by enhancing details by using transmittance according to claim 1, characterized in that: In step 3: The transmittance factor γ=e is introduced into the regional variance weight t(x) , to meet the detail extraction requirements of foggy images, the regional variance weight after the transmittance factor is introduced The definition is as follows: Where 1≤γ≤e, if x is located in the edge area, then Γ G (x) is usually greater than 1, then γ will have an effect on Γ G (x) is enhanced to a greater extent; if x is in the smooth region, then Γ G (x) is usually less than 1, then γ will have an effect on Γ G (x) weaken to a certain extent, The cost function E used by the weighted smoothing and sharpening filter is: in 1 / θ 2 ∈[0.01,100], η=κ'ε, solve We can get:

4. The method for defogging an aerial image by enhancing details using transmittance according to claim 1, characterized in that: In step 4: Weight ω k for: in, is a regularization factor to ensure s is a predefined scale parameter, and σ 2 is the average of the regional variances of the entire image.

Images