Aerial image defogging method using transmittance to enhance details

Abstract

The application discloses a kind of aerial photograph image defogging methods using transmissivity to enhance details, by solving atmospheric scattering model, image detail extraction and the fusion of the three parts of both.It is specifically by the regional search score method proposed to obtain atmospheric light value, while the transmissivity obtained by fog image after smoothing filtering is substituted into atmospheric scattering model, and defogging image solving is carried out.In image detail processing, weight smoothing sharpening filter operation is carried out to the smoothed image, and the transmissivity is introduced into the regional variance weight in the filter operation, and then the smoothing or sharpening operation is carried out by transmissivity control filter, the detail image is obtained by subtracting the smoothed image from the filtered image, and finally the defogging image is linearly fused from the image obtained by atmospheric scattering model and the image after detail processing.The original image details after defogging are effectively retained, and the clarity of the restored image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and particularly to a method for removing haze from aerial images using transmittance to enhance details. BACKGROUND

[0002] Aerial images have wide applications in air-to-ground target detection, while the quality of images collected in foggy weather is severely degraded, which has a serious impact on subsequent image applications such as target recognition. Therefore, image dehazing has important research value in improving the quality of images collected in foggy weather. At present, the methods for single image dehazing mainly include: image dehazing method based on atmospheric scattering model, image enhancement based method and deep learning based image dehazing. When the existing methods are used to dehaze images, there are problems such as improper solving of atmospheric light value causing halo and artifacts in the dehazed images, and image detail loss caused by atmospheric scattering model back solving. The present application aims to solve the problems of poor dehazing effect and detail loss in general image dehazing algorithm for aerial images in large scenes, and introduces transmittance in the regional variance solving to effectively preserve the near-far characteristics in image details and improve the quality of dehazed images.

[0003] To deal with the fog problem in images, HE et al. found that in most non-sky local regions, at least some pixels in a certain color channel have very low intensity, and proposed a single image dehazing method based on dark channel prior (He K, Sun J, Tang X. Single image haze removal using dark channel prior [C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009: 1956-1963). KANTI et al. divided the images into images with color cast and images without color cast, and performed color balancing through nonlinear transformation, and then performed fine adjustment based on color cast, thereby realizing image dehazing (Kanti Dhara S, Roy M, Sen D, et al. Colorcast dependent image dehazing via adaptive airlight refinement and non-linear color balancing [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(5): 2076-2081). SUMMARY

[0004] The application provides a haze removal method for aerial image with transmittance enhanced details, comprising the following steps:

[0005] Step 1: establishment of atmospheric scattering model,

[0006] The expression of the atmospheric scattering model of the image collected in the fog environment is as follows:

[0007] I(x) = J(x)t(x) + A(1-t(x)) (1)

[0008] Wherein, x represents a pixel point, I represents a fog image, J represents a haze-free image, that is, an image to be solved by haze removal, t(x) represents transmittance, and A represents atmospheric light value;

[0009] Step 2: obtaining of atmospheric light value and transmittance,

[0010] The atmospheric light value is obtained by the region score method, and the specific steps are as follows: first, the size of the light value search box is adjusted according to the size of the fog image, and the atmospheric light value is generally obtained by the quadtree method after 5-6 iterations, that is, the atmospheric light region size is 1 / 4 of the whole image 6 Therefore, the size of the atmospheric light value search box boxsize is determined as follows:

[0011]

[0012] Wherein, H and L are the height and width of the image, the search speed of the region box is further set, and finally the contrast and brightness comprehensive score of the region window is constructed to score the region covered by the search box;

[0013] In the selection of contrast, the Weber contrast established according to the human visual system is selected, and the Weber contrast C Weber is the normalized difference value of the object value and the background value, and the formula is as follows:

[0014]

[0015] Wherein, C Weber is the Weber contrast of the target point, I c.object is the pixel value of the target point in the image, and I c.ground is the average value of the pixels in the target point region;

[0016] The atmospheric light region score needs to consider the region contrast and the region brightness value, therefore, the maximum region value and the average region value are used as the brightness score, and the region score formula is as follows:

[0017]

[0018] Wherein, c∈{r,g,b} is the image red, green, blue three channels, B is the search box area, finally, in the highest score area Ω, the distance of each pixel point color vector and RGB maximum value is calculated, the pixel point which makes the distance minimum is selected as the estimated value of global atmospheric light value, that is:

[0019]

[0020] Wherein, x r is the value of pixel point x in red channel, x g is the value of pixel point x in green channel, x b is the value of pixel point x in blue channel, and the solving formula of transmittance is:

[0021]

[0022] Wherein A control factor δ=0.6 is introduced in the formula to constrain the rate of change;

[0023] The atmospheric scattering model formula (1) is solved by the atmospheric light value A(x) and transmittance t(x) obtained according to formula (5) and formula (6), that is, the dehazed image solved by the atmospheric scattering model is:

[0024]

[0025] Step 3: introduce the regional variance of transmittance to establish,

[0026] The local variance is introduced in the smoothing and sharpening filter technology as an edge perception weight to constitute a weight smoothing and sharpening filter, and the detail enhancement effect of the fog image is improved. The local regional variance weight Γ G (x) formula is as follows:

[0027]

[0028] Wherein N is the number of pixel points in the local region ξ, and x' is the pixel point in the region ξ;

[0029] The transmittance factor γ=e t(x) Is introduced in the regional variance weight to meet the detail extraction demand of the fog image. The regional variance weight after introducing the transmittance factor Is defined as follows:

[0030]

[0031] In the formula, 1≤γ≤e, if x is located in the edge region, then Γ G (x) is usually greater than 1, at this time γ will affect Γ G(x) is enhanced significantly; if x is in a smooth region, then Γ G (x) is usually less than 1, in which case γ will affect Γ G (x) is weakened to a certain extent.

[0032] The cost function E used in weighted smoothing and sharpening filtering is:

[0033]

[0034] in 1 / θ 2 ∈[0.01,100], η=κ'ε, solve We can obtain:

[0035]

[0036] Step 4: Introduce detailed transmittance extraction.

[0037] Because fog can distinguish between foreground and background in an image, simply enhancing the details of the entire foggy image would lead to image distortion. By leveraging the advantage of weighted smoothing and sharpening filtering, which can both smooth and sharpen, the background areas of the foggy image are smoothed, while the foreground areas are sharpened. (Definition) in To set a transmittance threshold, the transmittance distribution of a large number of foggy images was statistically analyzed. The median transmittance of most foggy images was between 0.7 and 0.8. When the value is small, the value of κ' in the foreground will be too large, leading to over-sharpening of the image; when... When the value is too large, it will cause the background portion of the image to have an excessively small value, resulting in a smooth image transition. Therefore, the value of κ' should be:

[0038]

[0039] When the transmittance is greater than the threshold (κ' > 1), the filter is adjusted to sharpening; when the transmittance is less than the threshold (κ' < 1), the filter is adjusted to smoothing. The final detail image is defined as follows:

[0040]

[0041] Where D(p) is the detail map of region ξ, and the weight ω k for:

[0042]

[0043] in, It is a regularization factor to ensure s is a predefined scale parameter, while σ 2It is the average of the regional variances of the entire image;

[0044] Step 5: Linear fusion image restoration.

[0045] The final dehazed image is obtained by linearly fusing the image solved by the atmospheric scattering model with the detail-sharpened image, that is:

[0046] R(x)=J(x)+D(x) (15)

[0047] Where R(x) is the final dehazed image, J(x) is the image obtained by solving formula (7), and D(x) is the detail image extracted in step 2.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] The method described in this invention addresses the problem of difficulty in accurately estimating atmospheric light values ​​in aerial images by proposing a region scoring method to obtain the optimal atmospheric light value. At the same time, current dehazing algorithms suffer from detail loss when processing aerial fog images, so a weighted smoothing-sharpening filtering algorithm is proposed to introduce transmittance into the region variance to sharpen the details of the dehazed image, thereby improving the clarity of the dehazed image. Attached Figure Description

[0050] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 The image comparison before and after processing in the embodiments of the present invention is as follows: where a is a naturally foggy image, b is a dehazed image using the dark channel prior method, c is a dehazed image using nonlinear color balance, and d is a dehazed image using the method of the present invention. Detailed Implementation

[0052] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific embodiments. However, the scope of protection of this invention is not limited to the content described.

[0053] Step 1: Establishing an atmospheric scattering model.

[0054] Images captured in foggy environments, such as Figure 1 As shown in Figure a, the expression for its atmospheric scattering model is:

[0055] I(x)=J(x)t(x)+A(1-t(x)) (1)

[0056] Where x represents a pixel, I represents a foggy image, J represents a fog-free image (i.e., the image obtained after defogging), t(x) represents transmittance, and A represents atmospheric illumination value.

[0057] Step 2: Determining atmospheric light value and transmittance.

[0058] Atmospheric illumination values ​​are obtained using the region scoring method. The specific steps are as follows: First, adjust the size of the light value search box according to the size of the foggy image. Atmospheric illumination values ​​are then obtained using the quadtree method, typically stopping after 5-6 iterations. That is, the size of the atmospheric illumination region is 1 / 4 of the entire image. 6 Therefore, the size of the atmospheric light value search box (boxsize) is determined as follows:

[0059]

[0060] Where H=600 and L=600 are the height and width of the image, the search speed of the region box is set to 4, and finally the contrast and brightness of the region pane are combined to score the area covered by the search box.

[0061] In selecting the contrast ratio, the Weber contrast ratio, based on the human visual system, is chosen. The Weber contrast ratio C is... Weber The normalized difference between the object value and the background value is calculated using the following formula:

[0062]

[0063] Among them, C Weber For the Weber contrast of the target point, I c.object I represents the pixel value of the target point in the image. c.ground This represents the average value of pixels within the target point area.

[0064] The atmospheric illumination area score needs to consider both area contrast and area brightness. Therefore, the maximum and average values ​​of the area are used as the brightness score. The area score formula is as follows:

[0065]

[0066] Where c∈{r,g,b} represents the red, green, and blue channels of the image, and B represents the search box region; finally, in the region Ω with the highest score, the distance between the color vector of each pixel and the maximum RGB value is calculated, and the pixel that minimizes this distance is selected as the estimated value of the global atmospheric illumination value, that is:

[0067]

[0068] Where, x r Let x be the value of pixel x in the red channel. gLet x be the value of pixel x in the green channel. b Let x be the value of pixel x in the blue channel; the formula for calculating transmittance is:

[0069]

[0070] in A control factor δ = 0.6 is introduced in the formula to constrain the rate of change;

[0071] The atmospheric light value A(x) and transmittance t(x) obtained from equations (5) and (6) can be used to solve the atmospheric scattering model formula (1), that is, the cloud and fog image obtained by the atmospheric scattering model is:

[0072]

[0073] Step 3: Introduce regional variance of transmittance.

[0074] In the smoothing and sharpening filtering technique, local variance is introduced as an edge-aware weight to form a weighted smoothing and sharpening filter, which improves the detail enhancement effect of foggy images. The local region variance weight Γ G The formula for (x) is as follows:

[0075]

[0076] in N is the number of pixels in the local region ξ. According to the formula (17), the boxsize is N = 81, and x' is the number of pixels in the region ξ.

[0077] A transmittance factor γ = e is introduced into the regional variance weighting. t(x) To meet the detail extraction requirements of foggy images, the region variance weights after introducing the transmittance factor are defined as follows.

[0078]

[0079] In the formula, 1 ≤ γ ≤ e. If x is located in the edge region, Γ(x) is usually greater than 1, and γ will significantly enhance Γ(x). If x is located in the smooth region, Γ(x) is usually less than 1, and γ will weaken Γ(x) to some extent. The final cost function E used for weighted smoothing and sharpening filtering is...

[0080]

[0081] in 1 / θ 2 =4, η=κ'ε, solve. achievable

[0082]

[0083] Step 4: Introduce detailed transmittance extraction.

[0084] Because fog can distinguish between foreground and background in an image, simply enhancing the details of the entire foggy image would lead to image distortion. This approach leverages the advantage of weighted smoothing and sharpening filtering, which can both smooth and sharpen, to smooth the background areas of a foggy image while sharpening the details of the foreground areas. (Definition...) in The set transmittance threshold was used; by statistically analyzing the transmittance distribution of a large number of foggy images, the median transmittance of most foggy images was 0.7–0.8. When the value is small, the value of κ' in the foreground will be too large, leading to over-sharpening of the image; when... When the value is too large, it will cause the background portion of the image to have an excessively small value, resulting in a poor smooth transition in the image; setting The value of κ' is

[0085]

[0086] When the transmittance is greater than 0.75, κ' > 1, and the filter is adjusted to sharpening; when the transmittance is less than 0.75, κ' < 1, and the filter is adjusted to smoothing; the final detail image D(p) is defined as follows:

[0087]

[0088] Where D(p) is the detail map of region ξ, and the weight ω k for:

[0089]

[0090] in, It is a regularization factor to ensure s is a predefined scaling parameter, set to 1, while σ 2 It is the average of the regional variances of the entire image;

[0091] Step 5: Linear fusion image restoration.

[0092] The final dehazed image is obtained by linearly fusing the image solved by the atmospheric scattering model with the detail-sharpened image, that is:

[0093] R(x)=J(x)+D(x) (15)

[0094] Where R(x) is the final dehazed image, such as Figure 1 As shown in d, J(x) is the image obtained by solving formula (7), and D(x) is the detail image extracted in step 2.

[0095] To further illustrate the advantages of the method of the present invention, Figure 1 Dehazed images obtained using the dark channel prior method described in the background technique are as follows: Figure 1 As shown in Figure b, the dehazed image obtained using the nonlinear color balance dehazing method in the background technique is as follows: Figure 1 As shown in c, by Figure 1 It can be seen that the clarity of the photos is significantly improved after dehazing using various methods. Among them, the clarity and effect of the photos processed by the nonlinear color balance dehazing method are slightly better than those processed by the dark channel prior method, while the clarity and effect of the photos processed by the method of this invention are significantly better than those processed by the dark channel prior method and the nonlinear color balance dehazing method.

[0096] The main technical features, basic principles, and related advantages of the present invention have been described above. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the concept or basic characteristics of the invention. Therefore, the above-described embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

[0097] Furthermore, it should be understood that although this specification describes various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for defogging an aerial image with transmittance-enhanced details, characterized in that, The method comprises the following steps: Step 1: establishment of an atmospheric scattering model The expression of the atmospheric scattering model for collecting images in a fog environment is as follows: I(x) = J(x)t(x) + A(1-t(x)) (1) Wherein, x represents a pixel point, I represents a fog image, J represents a fog-free image, that is, a fog-removed image, t(x) represents a transmittance, and A represents an atmospheric light value; Step 2: obtaining of the atmospheric light value and the transmittance The atmospheric light value is obtained by using a region score method, and the specific steps are as follows: First, the size of the light value search box is adjusted according to the size of the foggy image, and the atmospheric light value is obtained by the quadtree method, generally with 5-6 iterations to stop, i.e. the atmospheric light region size is 1 / 4 of the whole image 6 Therefore, the size of the atmospheric light value search box boxsize is determined as Wherein, H and L are the height and width of the image, a search speed of the region frame is set, and finally, a contrast and brightness comprehensive score of the region window is constructed to score the region covered by the search frame; Wherein, c is the red, green and blue three channels of the image, B is the search frame region, and finally, in the region Omega with the highest score, the distance between the color vector of each pixel point and the maximum value of RGB is calculated, and the pixel point with the minimum distance is selected as the estimated value of the global atmospheric light value, that is: Wherein, x r is the value of the pixel point x in the red channel, x g is the value of the pixel point x in the green channel, x b is the value of the pixel point x in the blue channel, and the solving formula of the transmittance is: wherein A control factor δ = 0.6 is introduced in the formula to constrain the rate of change; According to the atmospheric light value A(x) and the transmittance t(x) obtained from formula (5) and formula (6), the atmospheric scattering model formula (1) is solved, that is, the fog-removed image solved by the atmospheric scattering model is as follows: Step 3: establishment of a region variance of the transmittance In the smoothing sharpening filter technology, local variance is introduced as an edge perception weight to constitute a weight smoothing sharpening filter, so as to improve the detail enhancement effect of the fog image, and the local region variance weight Γ G (x) is as follows: wherein N is the number of pixels in the local region ξ, and x' is a pixel in the region ξ. Step 4: detail extraction of the transmittance Definitions wherein is a set transmittance threshold, the median of the transmittance of the foggy images is 0.7-0.8 by statistical method, and the value of K' is: When the transmittance is greater than the threshold value, that is, κ' > 1, the filter is adjusted to a sharpening filter; when the transmittance is less than the threshold value, that is, κ' < 1, the filter is adjusted to a smoothing filter, and finally, the definition of the detail image is as follows: where D(p) is a detail map of the region ξ, ω k is a weight; Step 5: linear fusion image restoration The final fog-removed image is obtained by linearly fusing the image solved by the atmospheric scattering model and the detail sharpening image, that is: R(x) = J(x) + D(x) (15) Wherein, R(x) is the final fog-removed image, J(x) is the image solved by formula (7), and D(x) is the detail image extracted in step 2.

2. The aerial image defogging method using transmittance enhanced details according to claim 1, wherein, In step 2: In the selection of the contrast, the Weber contrast established according to the human visual system is adopted, and the formula is as follows: where C Weber is the Weber contrast of the target point, I c.object is the pixel value of the target point in the image, I c.ground is the average value of the pixels in the target point area range; The maximum value of the region and the average value of the region are used as the region brightness score, and the formula of the region brightness score is as follows:

3. The method for defogging aerial image with enhanced details by transmittance according to claim 1, characterized in that, In step 3: A transmittance factor γ = e t(x) To meet the requirement of detail extraction of foggy images, the transmittance factor γ = e is defined as follows: In the formula, 1≤γ≤e, if x is located in the marginal region, then Γ G (x) is usually greater than 1, in which case γ will affect Γ G (x) is enhanced significantly; if x is in a smooth region, then Γ G (x) is usually less than 1, in which case γ will affect Γ G (x) is weakened to a certain extent. The cost function E used in the weight smoothing and sharpening filter is as follows: wherein 1 / θ 2 ∈ [0.01, 100], η = κ'ε, solve It can be obtained:

4. The method for defogging aerial image with enhanced details by transmittance according to claim 1, characterized in that, In step 4: Weight ω k is: wherein, is a regularization factor to ensure s is a predefined scale parameter, while 2 is the average of the area variance of the entire image.

Images