A deep learning dehazing method based on texture brightness feature enhancement

Abstract

This invention discloses a deep learning dehazing method based on texture and brightness feature enhancement. First, a deep learning dehazing network based on texture and brightness feature enhancement is constructed based on U-Net. The fog image is input into the deep learning dehazing network for texture and brightness feature enhancement. Downsampling and upsampling operations are then performed sequentially to obtain a dehazed image result, i.e., a higher-quality, clear, and fog-free image. The deep learning dehazing network described in the method utilizes multi-scale feature fusion, a visual transformation module, and discrete wavelet transform up- and downsampling modules to filter important information in the frequency domain and enhance the interactive fusion of underlying features. A new brightness texture guidance module is proposed, combining image detail and brightness information to guide the encoder's feature fusion process, thereby retaining richer semantic information in the output image. This method is independent of the specific fog image imaging relationship and can reconstruct the dehazed image. The dehazed image is of higher quality and clearer.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical image defogging, and specifically relates to a deep learning defogging method based on texture brightness feature enhancement. Background Art

[0002] With the rapid development of artificial intelligence (AI), drones have seen significant applications in agriculture, surveying and mapping, regional logistics, environmental monitoring, disaster relief, and intelligent transportation. Drones can carry visual sensors such as digital cameras and video cameras, enabling them to capture images and videos of real-world scenes through visual perception. However, in adverse weather conditions such as haze, drone aerial images suffer from low image quality and a lack of detailed target information, hindering subsequent processing such as target detection, target behavior analysis, and scene semantic analysis. Consequently, optical image dehazing technology is a current research hotspot.

[0003] Currently, there are three main types of image dehazing methods: (1) dehazing methods based on prior information; (2) dehazing methods based on image enhancement; and (3) dehazing methods based on deep learning. Among them, the dehazing methods based on prior information attempt to discover some prior laws between foggy images and clear images, and use these laws to guide the dehazing methods. When the quality of foggy images is severely degraded, this method cannot be used because it cannot provide accurate prior information. The dehazing methods based on image enhancement enhance the performance of hazy images by enhancing the fog-free areas in the image, but this will cause significant color distortion.

[0004] There are two approaches to deep learning-based dehazing: one is to learn the transmission map and atmospheric light intensity components of the fog-image relationship to calculate a haze-free image; the other is to directly estimate the dehazed image through an end-to-end design. The latter avoids the drawback of inaccurate descriptions of the fog-image relationship and is independent of the specific fog-image relationship. It utilizes a deep learning network to first extract image features at different scales through downsampling, and then generates the dehazed image through upsampling. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a deep learning defogging method based on texture brightness feature enhancement, which introduces enhanced information into the deep learning defogging network to obtain higher quality clear and fog-free images.

[0006] The technical solution adopted by the present invention is: a deep learning defogging method based on texture brightness feature enhancement, the specific steps are as follows:

[0007] S1. Build a deep learning dehazing network based on texture brightness feature enhancement based on U-Net;

[0008] The deep learning dehazing network includes: 1 texture brightness feature enhancement module, 6 visual transformation modules ViT-Block, 2 discrete wavelet transform downsampling modules DWT-Down, 1 neck network module BottleNeck, and 2 discrete wavelet transform upsampling modules DWT-Up.

[0009] S2. Input the fog image into the texture brightness feature enhancement module and output the fog image with enhanced texture brightness;

[0010] S3. Based on step S2, downsampling and upsampling operations are performed on the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module in sequence to obtain a defogging image result.

[0011] Furthermore, the step S2 is specifically as follows:

[0012] S21, extracting texture features from the fog image input to the texture brightness feature enhancement module;

[0013] Set the input image F in For a three-channel color image H×W×3, the improved adaptive threshold Canny algorithm is used to extract the edge texture information of the input image and obtain the texture mapping feature image F1 of the original image.

[0014] Where H represents the image height and W represents the image width.

[0015] First, a Gaussian kernel convolution operation is performed on the input image to smooth the input image and eliminate some abnormal high-frequency noise. The expression is as follows:

[0016]

[0017] Among them, I G represents the image after Gaussian smoothing, (x, y) represents the input image coordinates, (u, v) represents the coordinates of the Gaussian kernel, σ controls the size of the Gaussian convolution kernel, satisfying the kernel radius k≈3σ.

[0018] Then, the Sobel operator is used to calculate the gradient information of the smoothed image. The expression is as follows:

[0019]

[0020] Among them, * represents the convolution operation, D x and D y Represents the Sobel operator in the horizontal and vertical directions respectively. Using the gradient map in the xy direction, the amplitude information of the gradient map is calculated. Phase information

[0021] Next, we determine the strength of edge pixels. First, for each pixel, we check its gradient direction and compare the gradient amplitude information of adjacent pixels in the gradient direction. That is, we compare the gradient amplitude of the current pixel with the gradient amplitude of adjacent pixels in the gradient direction. If the gradient amplitude of the current pixel is not the local maximum, it is set to 0. We use the non-maximum suppression method to process it, retaining the local maximum gradient and suppressing the gradient of the non-edge part. The expression is as follows:

[0022]

[0023] Among them, Z represents the gradient amplitude map after maximum suppression processing, and G represents the original gradient amplitude map.

[0024] Then use double threshold division to divide the edge of the retained pixel area. The expression is as follows:

[0025]

[0026] Among them, T low and T high Respectively represent the minimum and maximum thresholds for determining whether the gradient amplitude is a strong or weak edge. If the current pixel gradient amplitude is greater than T high , it is determined to be a strong edge; if the current pixel gradient amplitude is less than T high But it is greater than T low , it is determined to be a weak edge; if the current pixel gradient amplitude is less than T low , it is judged as non-edge.

[0027] A method for local adaptive threshold adjustment is designed to refine the original global threshold setting through windowing operation. The expression is as follows:

[0028]

[0029] Where (m,n) represents the coordinates within the local window, g represents the gradient magnitude of each pixel in the window, and Dirac represents the Dirac function.

[0030] In the divided local window, the gradient magnitude histogram H(g) is calculated and the number of occurrences of each pixel magnitude in the current window is counted.

[0031] According to the gradient magnitude histogram H(g), a cumulative magnitude histogram C(g) is constructed, which represents the number of pixels in the current window whose gradient magnitude is greater than or equal to g. The expression is as follows:

[0032]

[0033] Based on the statistical information of the local histogram, the high and low thresholds used to judge the strength of the texture in the current window are calculated. The expression is as follows:

[0034] T high (i,j)=max{g|C(g)>η·C(0)} (7)

[0035] T low (i,j)=max{g|C(g)>η′·C(0)} (8)

[0036] Among them, T high (i,j) and T low (i, j) represents the high and low thresholds to be applied to the window at row i and column j in the image. C(0) represents the sum of all gradient histograms in the current window, and η and η′ represent proportional constants.

[0037] Starting from the strong edge pixel, trace the edge along the gradient direction. If the weak edge pixel is connected to the strong edge pixel, it is marked as an edge; if the weak edge pixel is not connected to any strong edge pixel, it is discarded.

[0038] Finally, a texture mapping feature map F1 of size H×W×1 is obtained.

[0039] S22, extracting brightness features from the fog image input to the texture brightness feature enhancement module;

[0040] The input image is processed to obtain the brightness prior information F2 of the three channels of the image.

[0041] First, multiple Gaussian kernels with different kernel sizes are applied to the input image for convolution to obtain a Gaussian pyramid composed of local brightness information at different scales. The brightness maps of each layer are weighted fused to obtain the global brightness information L combined with Gaussian smoothing filters of different scales. multi , the expression is as follows:

[0042]

[0043] Among them, red, green, and blue represent the three channels of the input image. s Represents the sth Gaussian convolution kernel. s represents the fusion weight of the sth convolution kernel. k represents the Gaussian kernel radius.

[0044] According to the color characteristics of the image, a processing method is designed to use the color variance as the adjustment basis to adaptively adjust the weights of the RGB channels. The expression is as follows:

[0045]

[0046] Among them, Var(·) represents the variance of RGB color channels, [ω R ,ω G ,ωB ] represents the weight coefficient of RGB three channels.

[0047] Calculate the color contrast in the image and adjust the brightness. The expression is as follows:

[0048] L color (i,j)=ω R R(i,j)+ω G G(i,j)+ω B ·B(i,j) (11)

[0049] Among them, L color (i, j) represents the result of color channel information integration, R(i, j), G(i, j) and B(i, j) represent F in The pixel values ​​of the R, G and B channels are calculated point by point, and the channel information is integrated based on the weight coefficients of formula (10).

[0050] At the same time, considering the initial brightness information and the brightness adjustment based on color information, the brightness image generated by formula (11) is gamma corrected to enhance the image details, and the pixel value of the output image is normalized to [0, 255]. The expression is as follows:

[0051]

[0052] Among them, L gamma represents the result after Gamma correction, β represents the weight coefficient, γ represents the Gamma correction coefficient, L min and L max That is L gamma F2 represents the brightness prior feature map, and the size of the brightness prior feature map is H×W×1.

[0053] S23: Fusing the texture features and brightness features obtained in steps S21 and S22, and concatenating them with the input fog image, and then performing convolution block processing to output a fog image with enhanced texture and brightness;

[0054] Integrate the edge and brightness feature maps of the input image, and use two learnable parameters ω1 and ω2 to obtain the fused feature map F3 = ω1·F1+ω2·F2.

[0055] Subsequently, the input image is concatenated with F3 in the channel dimension. First, a 1×1 convolution is performed to resize the number of channels to 3. Then, a depthwise convolution with a kernel size of 5, a step size of 1, and a padding of 2 is performed to generate a fusion information feature map F as the color information and texture brightness information. out , the expression is as follows:

[0056] Fout =DepthwiseConv 5×5 (Conv 1×1 (Concat(F3,F in ))) (14)

[0057] Furthermore, the step S3 is specifically as follows:

[0058] The downsampling operation process includes: 3 ViT-Blocks and 2 DWT-Downs; the upsampling operation includes: 3 ViT-Blocks and 2 DWT-Ups.

[0059] During the downsampling operation, the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module are first input into the first ViT-Block in the downsampling process to extract feature information of different scales, and then input into the first DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; the image processed by the first DWT-Down and the texture brightness fusion feature map are simultaneously input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; then the image processed by the second DWT-Down and the texture brightness fusion feature map are simultaneously input into the third ViT-Block, and after three ViT-Blocks, feature information of different scales is extracted.

[0060] The features extracted by the third ViT-Block in the downsampling process are then input into BottleNeck for processing. Through frequency domain auxiliary processing, the output image features are obtained and enter the upsampling process.

[0061] During the upsampling process, the image features output after BottleNeck are first input into the first ViT-Block in the upsampling process to extract feature information of different scales, and then input into the first DWT-Up together with the image processed by the second DWT-Down in the downsampling process to increase the image size by a factor of 2; the image processed by the first DWT-Up is then input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Up together with the image processed by the first DWT-Down in the downsampling process to increase the image size by a factor of 2; finally, the image processed by the second DWT-Up is input into the third ViT-Block to extract feature information of different scales, and the image size is restored to the original input size.

[0062] Finally, the upsampled image is fused with the original fog image to obtain the generated dehazed image result.

[0063] Furthermore, in step S1, the deep learning dehazing network is trained using a public dataset and a loss function is used for model training, specifically as follows:

[0064] In order to make the predicted image closer to the real clear image at the pixel level, one of the loss functions L s The expression is as follows:

[0065]

[0066] in, and Y represent the predicted image and true label, respectively.

[0067] Alignment in the spatial domain and the introduction of frequency signal learning in the network, the frequency domain L1 loss function is introduced to train the network, and the expression is as follows:

[0068]

[0069]

[0070] Where M and N represent the width and height of the amplitude spectrum respectively. out and A represent the amplitude spectra of the network predicted image and the target image respectively, P out and P represent the phase spectra of the network predicted image and the target image respectively. amp and L pha represent the amplitude and phase losses respectively.

[0071] The input image is introduced into the loss function to calculate the contrast loss. By comparing the features of the network output and the ground truth, the contrast loss function expression is as follows:

[0072]

[0073] Among them, a, p, n represent the network prediction image, the true value image and the input image respectively. i represents the i-th layer output of feature extraction, ω i represents the weight coefficient of the i-th layer, d L1 Represents the L1 loss calculation result, and ε represents an arbitrarily small constant to prevent division by zero errors.

[0074] The total loss function expression is as follows:

[0075]

[0076] in, They represent the corresponding loss function weights respectively.

[0077] Furthermore, in step S1, the evaluation indicators of the defogging processing effect of the deep learning defogging network are PSNR and SSIM, which are as follows:

[0078] (1) Peak signal-to-noise ratio (PSNR);

[0079]

[0080] Where MAX represents the maximum possible value of the image pixel value, and MSE represents the mean square error of the pixel values ​​between the original image and the reconstructed image.

[0081] (2) Structural similarity index (SSIM);

[0082]

[0083] Among them, μ x and μ y denote the average values ​​of the original image and the reconstructed image, σ x and σ y denote the standard deviation of the original image and the reconstructed image, σ xy represents the covariance between the original image and the reconstructed image, and C1 and C2 are constants used for stabilization.

[0084] Beneficial effects of the present invention: The method of the present invention first constructs a deep learning dehazing network based on texture brightness feature enhancement based on U-Net, inputs the fog image into the deep learning dehazing network for texture brightness feature enhancement, and performs downsampling and upsampling operations in sequence to obtain a generated dehazing image result, that is, a clearer and fog-free image with higher quality. The deep learning dehazing network described in the method of the present invention utilizes multi-scale feature fusion, visual transformation module, discrete wavelet transform up- and downsampling modules to filter important information in the frequency domain and enhance the interactive fusion of underlying features. By proposing a new brightness texture guidance module, the detail information and brightness information in the image are combined to guide the feature fusion process of the encoder, thereby retaining richer semantic information in the output image, and is independent of the specific fog image imaging relationship. The dehazing image can be reconstructed, and the dehazing image quality is higher and clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is a flowchart of a deep learning defogging method based on texture brightness feature enhancement of the present invention.

[0086] Figure 2 This is a diagram of the deep learning defogging network structure described in an embodiment of the present invention.

[0087] Figure 3 Flowchart of enhancing the texture brightness feature of fog image in the texture brightness feature enhancement module in an embodiment of the present invention.

[0088] Figure 4 This is a visual comparison diagram of the defogging effects of the method of the present invention and the existing method in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] The method of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0090] like Figure 1 As shown in the figure, a flow chart of a deep learning defogging method based on texture brightness feature enhancement of the present invention is shown in the figure. The specific steps are as follows:

[0091] S1. Build a deep learning dehazing network based on texture brightness feature enhancement based on U-Net;

[0092] like Figure 2 As shown, the deep learning dehazing network includes: 1 texture brightness feature enhancement module, 6 vision transformation modules (Vision Transformer Block, ViT-Block), 2 discrete wavelet transform down-sampling modules (Discrete Wavelet Transform Down-sampling, DWT-Down), 1 neck network module BottleNeck, and 2 discrete wavelet transform up-sampling modules (Discrete Wavelet Transform Up-Sampling, DWT-Up).

[0093] Among them, during the upsampling and downsampling process of the feature map, the ViT-Block module is used to extract feature information of different scales; during downsampling, the DWT-Down module separates the low-frequency domain and high-frequency domain information, and the low-frequency domain feature information is passed to the next level ViT-Block; during upsampling, the high-frequency domain features separated by the DWT-Down module are introduced to retain the high-frequency domain information during the image sampling process.

[0094] S2. Input the fog image into the texture brightness feature enhancement module and output the fog image with enhanced texture brightness;

[0095] S3. Based on step S2, downsampling and upsampling operations are performed on the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module in sequence to obtain a defogging image result.

[0096] like Figure 3 As shown, in this embodiment, the step S2 is specifically as follows:

[0097] S21, extracting texture features from the fog image input to the texture brightness feature enhancement module;

[0098] Set the input image F inFor a three-channel color image H×W×3, the improved adaptive threshold Canny algorithm is used to extract the edge texture information of the input image and obtain the texture mapping feature image F1 of the original image.

[0099] Where H represents the image height and W represents the image width.

[0100] First, a Gaussian kernel convolution operation is performed on the input image to smooth the input image and eliminate some abnormal high-frequency noise. The expression is as follows:

[0101]

[0102] Among them, I G represents the image after Gaussian smoothing, (x, y) represents the input image coordinates, (u, v) represents the coordinates of the Gaussian kernel, σ controls the size of the Gaussian convolution kernel, satisfying the kernel radius k≈3σ. This embodiment uses three Gaussian kernels with σ values ​​of 1, 2, and 3 for multi-scale smoothing, and adds them together after weighting to obtain the final result of the Gaussian kernel convolution operation.

[0103] Then, the Sobel operator is used to calculate the gradient information of the smoothed image. The expression is as follows:

[0104]

[0105] Among them, * represents the convolution operation, D x and D y Represents the Sobel operator in the horizontal and vertical directions respectively. Using the gradient map in the xy direction, the amplitude information of the gradient map is calculated. Phase information

[0106] In order to refine the texture, remove pixels that are not true edges or have weak edge information, and determine the strength of edge pixels, first for each pixel, check its gradient direction and compare the amplitude information of the gradient of adjacent pixels in the gradient direction, that is, compare the gradient amplitude of the current pixel with the gradient amplitude of adjacent pixels in the gradient direction; if the gradient amplitude of the current pixel is not the local maximum, it is set to 0. This retains the local maximum gradient and suppresses the gradient of the non-edge part, using the non-maximum suppression method for processing, the expression is as follows:

[0107]

[0108] Among them, Z represents the gradient amplitude map after maximum suppression processing, and G represents the original gradient amplitude map.

[0109] Then use double threshold division to divide the edge of the retained pixel area. The expression is as follows:

[0110]

[0111] Among them, T low and T high Respectively represent the minimum and maximum thresholds for determining whether the gradient amplitude is a strong or weak edge. If the current pixel gradient amplitude is greater than T high , it is determined to be a strong edge; if the current pixel gradient amplitude is less than T high But it is greater than T low , it is determined to be a weak edge; if the current pixel gradient amplitude is less than T low , it is judged as non-edge.

[0112] Since the threshold setting will greatly affect the edge texture judgment result, different high and low thresholds need to be set for different types and sizes of input images for judgment. However, if the input is an image with obvious regional differences, the set global judgment threshold may lose its original effect in certain areas of the image. To address this problem, a method of local adaptive threshold adjustment is designed. Through windowing operations, the original global threshold setting is refined to better adapt to more types of input images. The expression is as follows:

[0113]

[0114] Where (m,n) represents the coordinates in the local window, g represents the gradient magnitude of each pixel in the window, and Dirac represents the Dirac function.

[0115] In the divided local window, the gradient magnitude histogram H(g) is calculated and the number of occurrences of each pixel magnitude in the current window is counted.

[0116] According to the gradient magnitude histogram H(g), a cumulative magnitude histogram C(g) is constructed, which represents the number of pixels in the current window whose gradient magnitude is greater than or equal to g. The expression is as follows:

[0117]

[0118] According to the statistical information of the local histogram, the high and low thresholds used to judge the strength of the texture in the current window are calculated. C(0) represents the sum of all gradient histograms in the current window, and η and η′ are proportional constants.

[0119] T high (i,j)=max{g|C(g)>η·C(0)} (7)

[0120] T low (i,j)=max{g|C(g)>η′·C(0)} (8)

[0121] Among them, Thigh (i,j) and T low (i, j) represents the high threshold and low threshold to be applied in the window of row i and column j in the image. In this embodiment, η and η′ are set to 0.1 and 0.05 respectively.

[0122] Starting from the strong edge pixel, trace the edge along the gradient direction. If the weak edge pixel is connected to the strong edge pixel, it is marked as an edge; if the weak edge pixel is not connected to any strong edge pixel, it is discarded.

[0123] Finally, a texture mapping feature map F1 of size H×W×1 is obtained.

[0124] S22, extracting brightness features from the fog image input to the texture brightness feature enhancement module;

[0125] In order to preserve the brightness distribution information of the original image as much as possible in the processed image, the input image is processed to obtain the brightness prior information F2 of the three channels of the image.

[0126] First, convolution is performed on the input image using multiple Gaussian kernels of different kernel sizes, which can effectively reduce high-frequency noise while better preserving the edge information of the image. A Gaussian pyramid consisting of local brightness information at different scales is obtained, and the brightness maps of each layer are weighted fused to obtain the global brightness information L combined with Gaussian smoothing filters of different scales. multi The expression is as follows:

[0127]

[0128] Among them, red, green, and blue represent the three channels of the input image. s Indicates the sth Gaussian convolution kernel. In this embodiment, S is 3, that is, three convolution kernels with σ values ​​of 8, 16, and 32 are used to perform large-scale smoothing on the input image. s represents the fusion weight of the sth convolution kernel. k represents the Gaussian kernel radius.

[0129] Considering that the color information of the image will also affect the contribution of brightness in different areas, a processing method is designed based on the color characteristics of the image. Using the color variance as the adjustment basis, the weights of the RGB channels can be adaptively adjusted. The expression is as follows:

[0130]

[0131] Where Var(·) represents the variance of the RGB color channels. [ω R ,ω G ,ω B ] represents the weight coefficient of RGB three channels.

[0132] In order to emphasize the brightness contribution of a specific color, the color contrast in the image can be calculated and the brightness can be adjusted. The expression is as follows:

[0133] L color (i,j)=ω R R(i,j)+ω G G(i,j)+ω B ·B(i,j) (11)

[0134] Among them, R(i,j), G(i,j) and B(i,j) represent F in The pixel values ​​of R, G and B channels are integrated based on the weight coefficients of formula (10). color (i, j) represents the result of integrating color channel information.

[0135] At the same time, considering the initial brightness information and the brightness adjustment based on color information, the brightness image generated by formula (11) is gamma corrected to enhance the image details, and the pixel value of the output image is normalized to [0, 255]. The expression is as follows:

[0136]

[0137] Among them, L gamma represents the result after Gamma correction, β represents the weight coefficient, γ represents the Gamma correction coefficient, L min and L max That is L gamma F2 represents the brightness prior feature map, and the size of the brightness prior feature map is H×W×1.

[0138] S23: Fusing the texture features and brightness features obtained in steps S21 and S22, and concatenating them with the input fog image, and then performing convolution block processing to output a fog image with enhanced texture and brightness;

[0139] In order to integrate the edge and brightness feature maps of the input image, two learnable parameters ω1 and ω2 are used to obtain the fused feature map F3 = ω1·F1+ω2·F2.

[0140] Subsequently, the input image is concatenated with F3 in the channel dimension. First, a 1×1 convolution is performed to resize the number of channels to 3. Then, a depthwise convolution with a kernel size of 5, a step size of 1, and a padding of 2 is performed to generate a fusion information feature map F as the color information and texture brightness information. out , the expression is as follows:

[0141] F out =DepthwiseConv5×5 (Conv 1×1 (Concat(F3,F in ))) (14)

[0142] like Figure 2 As shown, in this embodiment, the step S3 is specifically as follows:

[0143] The downsampling operation process includes: 3 ViT-Blocks and 2 DWT-Downs; the upsampling operation includes: 3 ViT-Blocks and 2 DWT-Ups.

[0144] During the downsampling operation, the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module are first input into the first ViT-Block in the downsampling process to extract feature information of different scales, and then input into the first DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; the image processed by the first DWT-Down and the texture brightness fusion feature map are simultaneously input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; then the image processed by the second DWT-Down and the texture brightness fusion feature map are simultaneously input into the third ViT-Block, and after three ViT-Blocks, feature information of different scales is extracted.

[0145] The features extracted by the third ViT-Block in the downsampling process are then input into BottleNeck for processing. Through frequency domain auxiliary processing, the output image features are obtained and enter the upsampling process.

[0146] During the upsampling process, the image features output after BottleNeck are first input into the first ViT-Block in the upsampling process to extract feature information of different scales, and then input into the first DWT-Up together with the image processed by the second DWT-Down in the downsampling process to increase the image size by a factor of 2; the image processed by the first DWT-Up is then input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Up together with the image processed by the first DWT-Down in the downsampling process to increase the image size by a factor of 2; finally, the image processed by the second DWT-Up is input into the third ViT-Block to extract feature information of different scales, and the image size is restored to the original input size.

[0147] Finally, the upsampled image is fused with the original fog image to obtain the generated dehazed image result.

[0148] In this embodiment, in step S1, the deep learning dehazing network is trained using a public dataset, and a loss function is used for model training, specifically as follows:

[0149] In order to restore the real high-quality fog-free image, one of the simplest methods is to make the predicted image closer to the real clear image at the pixel level. Then one of the loss functions L s The expression is as follows:

[0150]

[0151] in, and Y represent the predicted image and true label, respectively.

[0152] In addition to alignment in the spatial domain, since the learning of frequency signals is introduced into the network, the frequency domain L1 loss function is considered to be introduced for network training. The expression is as follows:

[0153]

[0154]

[0155] Where M and N represent the width and height of the amplitude spectrum (phase spectrum) respectively. out and A represent the amplitude spectra of the network predicted image and the target image respectively, P out and P represent the phase spectra of the network predicted image and the target image respectively. amp and L pha represent the amplitude and phase losses respectively.

[0156] In order to improve the generalization ability of the model and prevent the network from overfitting the original input during the learning process, the input image is introduced into the loss function to calculate the contrast loss. By comparing the features of the network output and the ground truth, it can effectively guide the network to generate high-quality images that are closer to the ground truth. This makes the generated image visually closer to the real situation and richer in details.

[0157] The contrast loss function expression is as follows:

[0158]

[0159] Among them, a, p, n represent the network prediction image, the true value image and the input image respectively. i represents the i-th layer output of feature extraction, ω i represents the weight coefficient of the i-th layer, d L1 Represents the L1 loss calculation result, and ε represents an arbitrarily small constant to prevent division by zero errors.

[0160] By minimizing L c, which can make the network effectively close to the true value and increase the distinction from the input image, thereby more effectively guiding the network to generate high-quality images that are closer to the ground truth.

[0161] The total loss function expression is as follows:

[0162]

[0163] in, They represent the corresponding loss function weights, which are set to 0.1, 0.1 in this embodiment.

[0164] In this embodiment, in step S1, the evaluation indicators of the defogging processing effect of the deep learning defogging network are PSNR and SSIM, which are as follows:

[0165] In order to compare the dehazing effects of different methods, the following two indicators are used:

[0166] (1) Peak Signal-to-Noise Ratio (PSNR);

[0167]

[0168] Where MAX represents the maximum possible value of the image pixel value (255 for an 8-bit image), and MSE represents the mean square error of the pixel values ​​between the original image and the reconstructed image.

[0169] (2) Structural Similarity Index (SSIM);

[0170]

[0171] Among them, μ x and μ y denote the average values ​​of the original image and the reconstructed image, σ x and σ y denote the standard deviation of the original image and the reconstructed image, σ xy represents the covariance between the original image and the reconstructed image, and C1 and C2 are constants used for stabilization.

[0172] PSNR focuses more on the overall error level and is suitable for quantifying the global reconstruction error of an image, but it lacks in terms of perceptual quality. SSIM emphasizes the structural similarity of images and can better reflect the human visual system's perception of image quality, making it suitable for evaluating the perceptual quality of images.

[0173] This example also conducted a comparative validation study, conducting extensive experiments on the SOTS-Indoor and Dense-haze datasets to compare the dehazing performance of the proposed method with existing methods. The results showed that compared with other advanced dehazing methods, the proposed method achieved higher performance in terms of PSNR, SSIM, and other performance indicators. The details are as follows:

[0174] (1) Using the SOTS-Indoor dataset, the training set contains 13,990 pairs of foggy and non-fog images, and the test set contains 500 pairs of foggy and non-fog images. The comparison results of the dehazing effect are shown in Table 1.

[0175] Table 1

[0176]

[0177] (2) Using the DenseHaze dataset, the comparison of dehazing effects on dense fog images is shown in Table 2.

[0178] Table 2

[0179]

[0180] The defogging effect of the method of the present invention and the main method can be compared as shown in the figure Figure 4 As shown in the figure, GT represents the real fog-free image.

[0181] In summary, the deep learning dehazing network described in the method of the present invention utilizes multi-scale feature fusion, visual transformation module, discrete wavelet transform up- and down-sampling modules to filter important information in the frequency domain and enhance the interactive fusion of underlying features. By proposing a new brightness texture guidance module, the detail information and brightness information in the image are combined to guide the feature fusion process of the encoder, thereby retaining richer semantic information in the output image, and is independent of the specific fog image imaging relationship. The dehazed image can be reconstructed, and the dehazed image quality is higher and clearer.

[0182] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present invention, and it should be understood that the scope of the present invention is not limited to such specific descriptions and embodiments. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A deep learning dehazing method based on texture brightness feature enhancement, the specific steps are as follows: S1. Build a deep learning dehazing network based on texture brightness feature enhancement based on U-Net; The deep learning defogging network includes: 1 texture brightness feature enhancement module, 6 visual transformation modules ViT-Block, 2 discrete wavelet transform downsampling modules DWT-Down, 1 neck network module BottleNeck, and 2 discrete wavelet transform upsampling modules DWT-Up; S2. Input the fog image into the texture brightness feature enhancement module and output the fog image with enhanced texture brightness; S3, based on step S2, downsampling and upsampling operations are performed on the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module in sequence to obtain a defogging image result; The step S2 is specifically as follows: S21, extracting texture features from the fog image input to the texture brightness feature enhancement module; Set the input image F in For a three-channel color image H×W×3, the improved adaptive threshold Canny algorithm is used to extract the edge texture information of the input image and obtain the texture mapping feature image F1 of the original image; Where H represents the image height and W represents the image width; First, a Gaussian kernel convolution operation is performed on the input image to smooth the input image and eliminate some abnormal high-frequency noise. The expression is as follows: Among them, I G represents the image after Gaussian smoothing, (x, y) represents the input image coordinates, (u, v) represents the coordinates of the Gaussian kernel, σ controls the size of the Gaussian convolution kernel, satisfying the kernel radius k≈3σ; Then, the Sobel operator is used to calculate the gradient information of the smoothed image. The expression is as follows: Among them, * represents the convolution operation, D x and D y Represents the Sobel operator in the horizontal and vertical directions respectively; using the gradient maps in the xy directions obtained respectively, the amplitude information of the gradient map is calculated Phase information Then, the strength of edge pixels is judged. First, for each pixel, its gradient direction is checked, and the amplitude information of the gradient of adjacent pixels is compared in the gradient direction, that is, the gradient amplitude of the current pixel is compared with the gradient amplitude of adjacent pixels in the gradient direction; if the gradient amplitude of the current pixel is not the local maximum, it is set to 0; the non-maximum suppression method is used for processing, retaining the local maximum gradient and suppressing the gradient of the non-edge part. The expression is as follows: Among them, Z represents the gradient amplitude map after maximum suppression processing, and G represents the original gradient amplitude map; Then use double threshold division to divide the edge of the retained pixel area. The expression is as follows: Among them, T low and T high Respectively represent the minimum and maximum thresholds for determining whether the gradient amplitude is a strong or weak edge. If the current pixel gradient amplitude is greater than T high , it is determined to be a strong edge; if the current pixel gradient amplitude is less than T high But it is greater than T low , it is determined to be a weak edge; if the current pixel gradient amplitude is less than T low , it is determined to be non-edge; A method for local adaptive threshold adjustment is designed to refine the original global threshold setting through windowing operation. The expression is as follows: Where (m,n) represents the coordinates in the local window, g represents the gradient magnitude of each pixel in the window, and Dirac represents the Dirac function; In the divided local window, calculate the gradient amplitude histogram H(g) and count the number of occurrences of each pixel amplitude in the current window; According to the gradient magnitude histogram H(g), a cumulative magnitude histogram C(g) is constructed, which represents the number of pixels in the current window whose gradient magnitude is greater than or equal to g. The expression is as follows: Based on the statistical information of the local histogram, the high and low thresholds used to judge the strength of the texture in the current window are calculated. The expression is as follows: T high (i,j)=max{g|C(g)>η·C(0)} (7) T low (i,j)=max{g|C(g)>η′·C(0)} (8) Among them, T high (i,j) and T low (i, j) represents the high threshold and low threshold to be applied in the window of row i and column j in the image; C(0) represents the sum of all gradient histograms in the current window, and η and η′ represent proportional constants; Starting from the strong edge pixel, trace the edge along the gradient direction. If the weak edge pixel is connected to the strong edge pixel, mark it as an edge; if the weak edge pixel is not connected to any strong edge pixel, discard it. Finally, a texture mapping feature map F1 of size H×W×1 is obtained; S22, extracting brightness features from the fog image input to the texture brightness feature enhancement module; Process the input image to obtain the brightness prior information F2 of the three channels of the image; First, multiple Gaussian kernels with different kernel sizes are applied to the input image for convolution to obtain a Gaussian pyramid composed of local brightness information at different scales. The brightness maps of each layer are weighted fused to obtain the global brightness information L combined with Gaussian smoothing filters of different scales. multi , the expression is as follows: Among them, red, green, and blue represent the three channels of the input image respectively; guass s represents the sth Gaussian convolution kernel; α s represents the fusion weight of the sth convolution kernel; k represents the Gaussian kernel radius; According to the color characteristics of the image, a processing method is designed to use the color variance as the adjustment basis to adaptively adjust the weights of the RGB channels. The expression is as follows: Among them, Var(·) represents the variance of RGB color channels, [ω R ,ω G ,ω B ] represents the weight coefficient of RGB three channels; Calculate the color contrast in the image and adjust the brightness. The expression is as follows: L color (i,j)=ω R ·R(i,j)+ω G ·G(i,j)+ω B ·B(i,j) (11) Among them, L color (i, j) represents the result of color channel information integration, R(i, j), G(i, j) and B(i, j) represent F in The pixel values ​​of R, G and B channels are calculated point by point, and the channel information is integrated based on the weight coefficients of formula (10); At the same time, considering the initial brightness information and the brightness adjustment based on color information, the brightness image generated by formula (11) is gamma corrected to enhance the image details, and the pixel value of the output image is normalized to [0, 255]. The expression is as follows: Among them, L gamma represents the result after Gamma correction, β represents the weight coefficient, γ represents the Gamma correction coefficient, L min and L max That is L gamma The minimum and maximum values ​​in ; F2 represents the brightness prior feature map, and the size of the brightness prior feature map obtained is H×W×1; S23: Fusing the texture features and brightness features obtained in steps S21 and S22, and concatenating them with the input fog image, and then performing convolution block processing to output a fog image with enhanced texture and brightness; Integrate the edge and brightness feature maps of the input image and use two learnable parameters ω1 and ω2 to obtain the fused feature map F3 = ω1·F1+ω2·F2; Subsequently, the input image is concatenated with F3 in the channel dimension. First, a 1×1 convolution is performed to resize the number of channels to 3. Then, a depthwise convolution with a kernel size of 5, a step size of 1, and a padding of 2 is performed to generate a fusion information feature map F as the color information and texture brightness information. out , the expression is as follows: F out =DepthwiseConv 5×5 (Conv 1×1 (Concat(F3,F in ))) (14)。 2. The deep learning defogging method based on texture brightness feature enhancement according to claim 1 is characterized in that: The step S3 is specifically as follows: The downsampling operation process includes: 3 ViT-Blocks and 2 DWT-Down; the upsampling operation includes: 3 ViT-Blocks and 2 DWT-Up; During the downsampling operation, the fog image and the texture brightness fusion feature map obtained after processing by the texture brightness feature enhancement module are first input into the first ViT-Block in the downsampling process to extract feature information of different scales, and then input into the first DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; the image processed by the first DWT-Down and the texture brightness fusion feature map are simultaneously input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Down to reduce the image size by downsampling at a ratio of 1 / 2; then the image processed by the second DWT-Down and the texture brightness fusion feature map are simultaneously input into the third ViT-Block, and after three ViT-Blocks, feature information of different scales is extracted; The features extracted by the third ViT-Block in the downsampling process are then input into BottleNeck for processing. Through frequency domain auxiliary processing, the output image features are obtained and enter the upsampling process. During the upsampling process, the image features output after BottleNeck are first input into the first ViT-Block in the upsampling process to extract feature information of different scales, and then input into the first DWT-Up together with the image processed by the second DWT-Down in the downsampling process to increase the image size by a factor of 2; the image processed by the first DWT-Up is then input into the second ViT-Block to extract feature information of different scales, and then input into the second DWT-Up together with the image processed by the first DWT-Down in the downsampling process to increase the image size by a factor of 2; finally, the image processed by the second DWT-Up is input into the third ViT-Block to extract feature information of different scales, and the image size is restored to the original input size; Finally, the upsampled image is fused with the original fog image to obtain the generated dehazed image result.

3. The deep learning defogging method based on texture brightness feature enhancement according to claim 1 is characterized in that: In step S1, the deep learning dehazing network is trained using a public dataset and a loss function is used for model training, specifically as follows: In order to make the predicted image closer to the real clear image at the pixel level, one of the loss functions L s The expression is as follows: in, and Y represent the predicted image and true label respectively; Alignment in the spatial domain and the introduction of frequency signal learning in the network, the frequency domain L1 loss function is introduced to train the network, and the expression is as follows: Where M and N represent the width and height of the amplitude spectrum respectively; A out and A represent the amplitude spectra of the network predicted image and the target image respectively, P out and P represent the phase spectra of the network predicted image and the target image respectively; L amp and L pha represent the amplitude and phase losses respectively; The input image is introduced into the loss function to calculate the contrast loss. By comparing the features of the network output and the ground truth, the contrast loss function expression is as follows: Among them, a, p, n represent the network prediction image, the true value image and the input image respectively; f i represents the i-th layer output of feature extraction, ω i represents the weight coefficient of the i-th layer, d L1 Represents the L1 loss calculation result, ε represents an arbitrarily small constant to prevent division by zero errors; The total loss function expression is as follows: in, They represent the corresponding loss function weights respectively.

4. The deep learning defogging method based on texture brightness feature enhancement according to claim 1 is characterized in that: In step S1, the evaluation indicators of the defogging processing effect of the deep learning defogging network are PSNR and SSIM, which are as follows: (1) Peak signal-to-noise ratio (PSNR); Where MAX represents the maximum possible value of the image pixel value, and MSE represents the mean square error of the pixel values ​​between the original image and the reconstructed image; (2) Structural similarity index (SSIM); Among them, μ x and μ y denote the average values ​​of the original image and the reconstructed image, σ x and σ y denote the standard deviation of the original image and the reconstructed image, σ xy represents the covariance between the original image and the reconstructed image, and C1 and C2 are constants used for stabilization.

Images