An uneven dehazing method combining LSD secondary segmentation and deep learning

Abstract

The present invention discloses an uneven defogging method combining LSD secondary segmentation and deep learning, belonging to the field of image processing technology. It is characterized in that the present invention is carried out through the following steps: step 1, obtaining a haze image data set; step 2, LSD secondary segmentation; step 3, performing light haze processing and dense haze processing; step 4, histogram correction; before splicing, histogram correction is performed on all image blocks to ensure that the colors between reconstructed image blocks are consistent; step 5, image splicing; step 6, eliminating the splicing seams by improving median filtering and total variation smoothing; finally, a defogging image is obtained. The present invention mainly solves the problem that the uneven degree of foggy images cannot be uniformly processed, and in the problem of defogging unevenness, many image details will be lost in terms of halo and block artifacts, and the authenticity of the picture will be lost.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an uneven defogging method combining LSD secondary segmentation and deep learning. Background Art

[0002] In recent years, environmental pollution has caused a lot of haze weather. In foggy weather, visibility is low, objects and the environment are blurred, distant objects are difficult to identify, image details and edge information are easily lost, and the uneven distribution of haze leads to reduced image quality in foggy environments. Therefore, in order to obtain clear images, researchers have conducted extensive research on image dehazing methods.

[0003] At present, image dehazing methods are mainly divided into two categories. The first category is dehazing algorithms based on prior knowledge. For example, Fattal estimates the transmission map based on the prior knowledge that there is no local statistical correlation between the shadow and the transmission map of the object surface. However, for dehazing methods based on prior knowledge, the recovery time is long and cannot meet the real-time requirements of the industry. Moreover, dehazing algorithms that rely on various prior knowledge cannot meet the needs of all scenarios. The second category is dehazing algorithms based on deep learning. For example, Li et al. first proposed an end-to-end dehazing model constructed by CNN, called the All-in-One Dehazing Network (AOD-Net). Its main advantage is that it can directly obtain a haze-free result from a single blurred image through CNN. The disadvantage is that it cannot handle dense haze, requires a large number of samples, and takes a long time to train.

[0004] All of the above methods have some problems. They cannot handle the unevenness of foggy images uniformly. In the problem of defogging unevenness, many image details will be lost in terms of halo and block artifacts, which loses the authenticity of the image. Summary of the invention

[0005] Purpose of the invention: In order to provide an improvement on uneven haze images, the present invention provides a defogging method combining LSD secondary segmentation and deep learning.

[0006] Technical solution:

[0007] An uneven defogging method combining LSD secondary segmentation and deep learning, characterized in that it includes the following steps:

[0008] Step 1: Obtain a haze image dataset;

[0009] Step 2, LSD secondary segmentation; first, calculate the difference in brightness and saturation at the coordinates of each point in the original image, set the initial difference threshold as the average difference between the point with the largest difference and the point with the smallest difference in the image, segment the areas with thicker haze and thinner haze respectively, and segment them into mn blocks. During the segmentation process, continuously update the preset threshold; perform further segmentation on the basis of segmentation, so that the difference in each small block is within a certain range, so as to ensure that the haze of each small block after segmentation is uniform;

[0010] Step 3: Perform haze and fog processing. For haze processing, an encoder and decoder are built on U-Net. The encoder has four convolution blocks, each of which contains two 3×3 convolution filter chains, a batch normalization layer, and a ReLU activation function. A maximum pooling operation is performed after each convolution block. The decoder includes four amplification blocks, each of which includes a pixel shuffle layer, followed by a 3×3 convolution filter, batch normalization, and ReLU activation. The result of the decoder is passed to a convolution layer with a filter size of 1×1 to obtain a clear image.

[0011] Dense fog processing adopts a weighted average image fusion strategy to adaptively improve the accuracy of transmission map estimation; assigns a larger weight to the initial transmission map and a lower weight to the gamma-corrected version; fuses the initial transmission map with the transmission map obtained through DCP and gamma correction to adaptively improve the accuracy of transmission map estimation; then guides the filtering of the fused transmission map of each image; finally, the defogged image block is obtained through the atmospheric scattering model I(X)=J(X)T(X)+A(1-T(X)); in the formula, I(X) represents the foggy image, J(X) represents the fog-free image, T(X) represents the transmission map, and A represents the atmospheric light.

[0012] Step 4: Histogram correction: Before stitching, all image blocks are subjected to histogram correction to ensure that the colors of the reconstructed image blocks are consistent;

[0013] Step 5, image stitching; restore the disordered image blocks by comparing the similarity of histograms; extract and save the left and right edges of the block images, compare the left and right edges in pairs, perform stitching operations on the block images whose similarity exceeds the preset threshold, and obtain the left and right stitched images; extract the upper and lower edges of the left and right stitched block images, compare the upper and lower edges in pairs, and perform stitching operations on the block images whose similarity exceeds the preset threshold to obtain a fog-free image;

[0014] Step 6. Eliminate the seams by improving the median filter and total variation smoothing. Improve the median filter and use the set threshold to guide the grayscale correction on both sides of the seams. This method only corrects the pixels on both sides that meet the set conditions, and the pixels that do not meet the conditions remain unchanged. The grayscale difference at the seams is weakened. The total variation smoothing method is used to further process the image after median filtering to ensure that the edge details are not affected. Finally, a dehazed image is obtained.

[0015] Preferably, in step 1, the RESIDE dataset is used. The RESIDE dataset is a synthetic dataset consisting of five subsets: an indoor training set, an outdoor training set, a comprehensive target test set, a real-world task-driven test set, and a mixed subjective test set; the indoor training set, the outdoor training set, and the comprehensive target test set are synthetic datasets, the real-world task-driven test set is a real dataset, and the mixed subjective test set is a dataset consisting of synthetic and real blurred images.

[0016] Preferably, in the LSD secondary segmentation in step 2, the following steps are performed:

[0017] Input the data set in step 1 and segment each data set;

[0018] 2.1 Define the coordinates of each point. According to the color attenuation prior, the difference between brightness and saturation is proportional to the division. The larger the difference, the higher the concentration; the smaller the difference, the lower the concentration;

[0019] 2.2 According to the characteristics of the image, the image is divided into m*n parts, that is, mn blocks. The center of each block is selected as the center point, with a total of mn center points. The difference c(x) between the brightness and saturation of each center point is calculated, and the average difference between the maximum and minimum of the original image is set as the initial threshold, that is:

[0020] c(x)=l(x)-s(x)

[0021]

[0022] Wherein, l(x) represents brightness, s(x) represents saturation, h represents a preset threshold, and x represents the pixel coordinates.

[0023] 2.3 If the difference of each coordinate of the block is greater than or equal to or less than the preset threshold, then Φ(x) is expanded outward until the maximum possible range is within the preset threshold, that is:

[0024] Assume that the range of each block after the first division is s;

[0025] If any c(x)≥h or c(x)≤h, its range expands outward by Φ(x);

[0026] 2.4 If the difference of a certain coordinate of the block is not within the preset threshold range, Φ(x) is gradually reduced until the maximum possible range is within the preset threshold range; that is:

[0027] In s, if there is Then its range shrinks inward by Φ(x);

[0028] 2.5 The preset threshold will be gradually updated according to the division of each block;

[0029] 2.6 Adaptively segment each block after segmentation, and the difference of each small block is less than or equal to r, so as to ensure that the haze of each small block is relatively uniform; that is:

[0030] Suppose the range of each small block after the second division is s'

[0031] For any s', c max (x)-c min (x)≤r; r represents the maximum range of the maximum difference and the minimum difference in each small block.

[0032] 2.7 Divide each small block into thick fog block and thin fog block.

[0033] Preferably, in the step of performing haze processing and dense fog processing in step 3, the U-Net encoder-decoder structure removes haze; first, based on the encoder-decoder architecture U-Net, the encoder uses 4 convolution blocks to represent the noisy input image into a latent space, and these convolution blocks extract relevant features at different scales, each convolution block includes 2 convolution filter chains of size 3×3, batch normalization layer and ReLU activation function, and a maximum pooling operation is performed after each convolution block to aggregate features and increase the receptive field size of subsequent convolution blocks; to reconstruct a haze-free image, 4 magnification blocks are used, each magnification block includes a pixel shuffle layer, followed by a convolution filter of size 3×3, batch normalization and ReLU activation; the features obtained at each level of the encoder are connected to the features of the corresponding level at the decoder end through a long jump connection; the result of the decoder is passed to a convolution layer with a filter size of 1×1, thereby obtaining a clear image; in the case of non-uniform haze, the affected area may exceed the perception of the convolution kernel, and LSD secondary segmentation is used to evenly divide the blocks, thereby solving the problem of inaccurate haze recovery;

[0034] Dense fog is processed by accurate transmission map; for the segmented dense fog, the initial transmission map is obtained by DCP Among them, t 1 (x) represents the initial transmission image, r, g, b represent the three color channels of red, green, and blue, A c represents the atmospheric light of any channel, I c(y) represents any color channel of I, Ω(x) represents the window at pixel x, and ω is a parameter. Gamma correction is introduced to preprocess the transmission map obtained by DCP. DCP means that in each local area of ​​most outdoor fog-free images, at least one color channel has a very low intensity value, thereby obtaining the initial transmission map; the transmission map formula with gamma correction is is the gamma value; the image corrected by DCP gamma and the initial transmission image are fused through an adaptive fusion strategy. The adaptive fusion strategy can maintain the details of the restored image and can adaptively and effectively fuse the image with different degrees of blur. The fusion formula is:

[0035] t f (x) = t 1 (x)w f (x)+t 2 (x)(1-w f (x)); where t 1 (x) is the initial transmission image, t 2 (x) is the transmission image after gamma correction, t f (x) is the fused transmission image, w f (x) is the fusion weight;

[0036] The transmission map and the gamma-corrected transmission map are filtered, namely:

[0037] g 1 (x) = guidf τ,ε (t 1 (x));

[0038] g 2 (x) = guidf τ,ε (t 2 (x));

[0039] τ, ε are the parameters of the guided filter, thus we get:

[0040]

[0041] in k is a constant;

[0042] The fused transmission image is guided filtered to obtain better visual effects and solve the halo and block artifact problems of the dehazed image, namely:

[0043] t f (x) = guidf τ,ε (I gray (x),t f (x));

[0044] Among them Igray (x) is the grayscale image of the original image;

[0045] Then through To achieve image defogging, I(x) in the formula represents the foggy image, J(x) represents the fog-free image, A represents the atmospheric light, and t f (x) is the transmission image after fusion.

[0046] Preferably, in the histogram correction of step 4, the following steps are performed:

[0047] 4.1 Convert color images to grayscale images and list the grayscale levels of each image;

[0048] 4.2 Calculate the probability of each histogram, that is, the proportion of pixels of each gray level in each image to the total, and select the histogram of the image with the best effect as the specified histogram;

[0049] 4.3 Calculate the cumulative histogram of each image;

[0050] 4.4 Calculate the absolute value of the difference between the cumulative histogram of each histogram and the cumulative histogram of the specified histogram;

[0051] 4.5 Construct a grayscale mapping table and determine the mapping relationship;

[0052] 4.6 Map each image grayscale to a new grayscale.

[0053] Preferably, in the image stitching of step 5, the image stitching is performed according to the following steps:

[0054] Image stitching is performed by histogram similarity contrast. The higher the similarity, the greater the possibility of stitching.

[0055] 5.1 Extract and save the left and right edges of the image;

[0056] 5.2 stitch the images whose similarity exceeds the preset threshold to obtain the left and right stitched images;

[0057] 5.3 Extract the upper and lower edges of the left and right spliced ​​image blocks;

[0058] 5.4 stitching the images whose similarity exceeds the preset threshold; finally obtaining a haze-free image;

[0059] If the similarity value reaches 0.95, the edges of the two images are considered similar and are stitched together.

[0060] Preferably, in step 6, in order to eliminate the seam by improving the median filter and total variation smoothing, the grayscale mean within the range Z on both sides of the seam is counted row by row, and then the grayscale values ​​of the pixels in each row within the range of the correction width W are sorted by median to obtain the grayscale median Mi , W needs to be less than Z; i represents the number of rows; set a threshold value, based on experience, take 0.02; use the pixel points within the range of W and the corresponding grayscale mean S i If the absolute value of the difference is greater than or equal to the threshold, the gray value of the point is modified to the corresponding M i , if the difference is less than the threshold, no change is made.

[0061] Total variation smoothing eliminates the seams. The total variation smoothing model is obtained by solving the Euler-Lagrange equation using the gradient descent method: is the diffusion coefficient. In the area where the grayscale changes strongly, the gradient value |▽μ| is large and the diffusion coefficient is small, so the diffusion is weak in the edge area and the image details can be retained. In the area where the grayscale changes gently, the gradient value |▽μ| is small and the diffusion coefficient is large, so the diffusion ability is strong in the flat area, solving the problem of smooth transition edges.

[0062] Beneficial effect: The present invention discloses a method for processing the uneven degree of haze in a foggy image by dividing it into blocks, and proposes an LSD secondary segmentation method. The original image is divided into thick fog blocks and thin fog blocks, so that the fog concentration of each block is relatively uniform. For thin fog, we use the U-Net encoder-decoder structure to obtain a clear image block. For thick fog, firstly, the transmission map t is obtained by DCP. 1 , t 1 Through gamma correction, we get t 2 , t 1 and t 2 Fusion, get clear image blocks through the atmospheric scattering model. Image defogging is achieved through image stitching method and improved median filtering and total variation smoothing.

[0063] 1. This method proposes an LSD secondary segmentation method, which flexibly divides the foggy image into blocks according to the degree of unevenness to ensure local relative uniformity. The existing DMPHN method divides the foggy image into blocks uniformly. Therefore, the results obtained by this method will have better performance than the existing methods in defogging with uneven degrees.

[0064] 2. This method uses image stitching and restoration. Before stitching, histogram correction is used to make the colors of each block consistent. After stitching, an improved median filter is used to reduce the grayscale difference at the stitching seams, and then total variation smoothing is used to further process the image to avoid excessive traces after segmentation and improve the visual effect.

[0065] 3. This method establishes a connection between the system and the existing defogging methods. It not only solves the problem of uneven defogging, but also achieves some results in terms of halo and block artifacts, retains image details, and has better results than existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Schematic diagram of the composition of the uneven defogging method of the present invention;

[0067] Figure 2 This is a diagram of mist processing based on the codec structure of the present invention;

[0068] Figure 3 This is a dense fog processing diagram based on the accurate transmission diagram of the present invention;

[0069] Figure 4 It is an algorithm flow chart of the uneven defogging method of the present invention;

[0070] Figure 5 This is the LSD secondary segmentation flow chart. DETAILED DESCRIPTION

[0071] The present invention will be described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0072] like Figure 1 , Figure 4 As shown in the figure: Since the existing methods are not very effective in removing uneven haze images, a method for secondary segmentation of haze according to the degree of unevenness is designed to complete the segmentation of dense fog and light fog areas of uneven foggy images. Because the difference between brightness and saturation is positively correlated, the larger the difference, the higher the fog concentration; the smaller the difference, the lower the fog concentration. Based on this principle, the difference is used to divide the uneven foggy image into uniform dense fog blocks and thin fog blocks through two segmentations. The first segmentation is divided into thin fog and dense fog according to the preset threshold, and then the image after the first segmentation is segmented again to make the fog concentration of each small block more uniform. The segmented dense fog and thin fog are processed separately, and the thin fog is processed by the U-Net encoder-decoder structure. The dense fog is fused according to the initial transmission map and the transmission map after DCP and gamma correction. The processed dense fog and thin fog are first histogram corrected and then spliced, and the fog-free image is reconstructed through improved median filtering and total variation smoothing.

[0073] The specific steps are as follows:

[0074] Step 1: Obtain a haze image dataset;

[0075] We use the RESIDE dataset for our experiments. RESIDE is a widely used synthetic dataset that consists of five subsets: Indoor Training Set (ITS), Outdoor Training Set (OTS), Synthetic Objective Test Set (SOTS), Real World Task Driven Test Set (RTTS), and Hybrid Subjective Test Set (HSTS). ITS, OTS, and SOTS are synthetic datasets, RTTS is a real-world dataset, and HSTS consists of synthetic and real blurry images.

[0076] Step 2, LSD secondary segmentation;

[0077] Input the data set in step 1 and split each data set. Figure 5 shown.

[0078] 2.1 Define the coordinates of each point, and divide it proportionally to the difference in brightness and saturation according to the color attenuation prior. The larger the difference, the higher the concentration; the smaller the difference, the lower the concentration.

[0079] 2.2 According to the characteristics of the image, the image is divided into m*n parts, that is, mn blocks. The center of each block is selected as the center point, a total of mn center points, and the difference c(x) between the brightness and saturation of each center point is calculated. The average difference between the maximum and minimum of the original image is set as the initial threshold. That is:

[0080] c(x)=l(x)-s(x)

[0081]

[0082] Wherein, l(x) represents brightness, s(x) represents saturation, h represents a preset threshold, and x represents pixel coordinates.

[0083] 2.3 If the difference of each coordinate of the block is greater than or equal to or less than the preset threshold, then Φ(x) is expanded outward until the maximum possible range is within the preset threshold. That is:

[0084] Assume that the range of each block after the first division is s

[0085] If any c(x)≥h or c(x)≤h, its range will expand outward by Φ(x).

[0086] 2.4 If the difference of a certain coordinate of the block is not within the preset threshold range, Φ(x) is gradually reduced until the maximum possible range is within the preset threshold range. That is:

[0087] In s, if there is Then its range shrinks inward by Φ(x).

[0088] 2.5 The preset threshold will be gradually updated according to the division of each block.

[0089] 2.6 Adaptively segment each block after segmentation, and the difference of each small block is less than or equal to r, so as to ensure that the haze of each small block is relatively uniform. That is:

[0090] Suppose the range of each small block after the second division is s'

[0091] For any s', c max (x)-c min (x)≤r; r represents the maximum range of the maximum difference and the minimum difference in each small block.

[0092] 2.7 Divide each small block into thick fog block and thin fog block.

[0093] Step 3: U-Net encoder-decoder structure removes haze;

[0094] like Figure 2 Figure 1: To restore the areas affected by haze, we build on the encoder-decoder architecture U-Net. The encoder represents the noisy input image into a latent space using 4 convolutional blocks that extract relevant features at different scales. Each convolutional block consists of 2 chains of convolutional filters of size 3×3, batch normalization layers, and ReLU activation functions. A max pooling operation is performed after each convolutional block to aggregate features while increasing the receptive field size of subsequent convolutional blocks. To reconstruct the haze-free image, 4 upscaling blocks are used, each consisting of a pixel shuffle layer followed by a convolutional filter of size 3×3, batch normalization, and ReLU activation. The features obtained at each level of the encoder are connected to the features of the corresponding level at the decoder end through long skip connections. This ensures that the fine-grained features extracted in the early layers are present in the noise-free image and helps preserve the boundary properties of the objects present in the image. The results of the decoder are passed to a convolutional layer with a filter size of 1×1. This results in a clear image. In the case of non-uniform haze, the affected area may exceed the receptive field of the convolution kernel, resulting in weak representations extracted by the encoder along different scales. Therefore, we use LSD secondary segmentation to evenly divide the blocks to solve the problem of inaccurate haze recovery.

[0095] Step 4: Processing dense fog through accurate transmission map;

[0096] like Figure 3 As shown: For the segmented dense fog, the initial transmission map is obtained through DCP Among them, t 1 (x) represents the initial transmission image, r, g, b represent the three color channels of red, green, and blue, A c represents the atmospheric light of any channel, I c (y) represents any color channel of I, Ω(x) represents the window at pixel x, and ω is a parameter. Gamma correction is introduced to preprocess the transmission map obtained by DCP. DCP means that in each local area of ​​most outdoor fog-free images, at least one color channel has a very low intensity value, so as to obtain the initial transmission map. The transmission map formula with gamma correction is Gamma

[0097] Value. An adaptive fusion strategy is used to fuse the image after DCP gamma correction and the initial transmission image. The adaptive fusion strategy can maintain the details of the restored image and can adaptively perform effective fusion for different degrees of image blur. The fusion formula is:

[0098] t f (x) = t 1 (x)w f (x)+t 2 (x)(1-w f (x))

[0099] where t 1 (x) is the initial transmission image, t 2 (x) is the transmission image after gamma correction, w f (x) is the fusion weight, t f (x) is the transmission image after fusion.

[0100] The adaptive fusion strategy can maintain the details of the restored image and can adaptively fuse effectively for different degrees of image blur. The fused transmission map is processed by guided filtering to solve the problems of transition enhancement and color distortion. The transmission map and the gamma-corrected transmission map are filtered. That is:

[0101] g 1 (x) = guidf τ,ε (t 1 (x)

[0102] g 2 (x) = guidf τ,ε (t 2 (x))

[0103] τ, ε are the parameters of the guided filter. Thus we get

[0104]

[0105] in k is a constant;

[0106] The fused transmission image is guided and filtered to obtain better visual effects and solve the halo and block artifact problems of the dehazed image. That is:

[0107] t f (x) = guidf τ,ε (I gray (x),t f (x))

[0108] Among them I gray (x) is the grayscale image of the original image.

[0109] Then through Image defogging is achieved. In the formula, I(x) represents the foggy image, J(x) represents the fog-free image, A represents the atmospheric light, and t f (x) is the transmission image after fusion.

[0110] Step 5: Make the colors of each block consistent through histogram correction

[0111] All processed image blocks are equalized to form the same normalized histogram, and then the original image is inversely equalized using the uniform histogram as a medium to ensure that the color of the image remains consistent.

[0112] 5.1 Convert color images to grayscale images and list the grayscale levels of each image;

[0113] 5.2 Calculate the probability of each histogram, that is, the proportion of pixels of each gray level in each image to the total. Select the histogram of the image with the best effect as the specified histogram;

[0114] 5.3 Calculate the cumulative histogram of each image;

[0115] 5.4 Calculate the absolute value of the difference between the cumulative histogram of each histogram and the cumulative histogram of the specified histogram;

[0116] 5.5 Construct a grayscale mapping table and determine the mapping relationship;

[0117] 5.6 Map each image grayscale to a new grayscale;

[0118] Step 6: stitch images;

[0119] Image stitching is performed by histogram similarity contrast. The higher the similarity, the greater the possibility of stitching.

[0120] 6.1 Extract and save the left and right edges of the image,

[0121] 6.2 The images whose similarity exceeds the preset threshold are stitched together to obtain the left and right stitched images.

[0122] 6.3 Extract the upper and lower edges of the left and right spliced ​​image blocks.

[0123] 6.4 The images whose similarity exceeds the preset threshold are stitched together to finally obtain a haze-free image.

[0124] It is worth noting that: according to experimental results, if the similarity value reaches 0.95, the edges of the two images are considered to be similar and can be spliced.

[0125] Step 7: Eliminate seams by improving median filtering and total variation smoothing

[0126] The grayscale mean within the range Z on both sides of the seam is counted row by row, and then the grayscale values ​​of the pixels in each row within the range of the correction width W are sorted by median to obtain the grayscale median M i , W needs to be less than Z. Where i represents the number of rows. Set a threshold value, based on experience, and take 0.02. Use the pixel points within the range of W and the corresponding grayscale mean S i If the absolute value of the difference is greater than or equal to the threshold, the gray value of the point is modified to the corresponding M i , if the difference is less than the threshold, no change is made.

[0127] Total variation smoothing eliminates the seams. The total variation smoothing model is obtained by solving the Euler-Lagrange equation using the gradient descent method: is the diffusion coefficient. In the area where the grayscale changes strongly, the gradient value |▽μ| is large and the diffusion coefficient is small, so the diffusion is weak in the edge area and the image details can be retained. In the area where the grayscale changes gently, the gradient value |▽μ| is small and the diffusion coefficient is large, so the diffusion ability is strong in the flat area, solving the problem of smooth transition edges.

[0128] Attached variable table:

[0129]

Claims

1. An uneven dehazing method combining LSD secondary segmentation and deep learning, Features: The following steps are involved: Step 1: Obtain a haze image dataset; Step 2, LSD secondary segmentation; first, calculate the difference in brightness and saturation at the coordinates of each point in the original image, set the initial difference threshold as the average difference between the point with the largest difference and the point with the smallest difference in the image, segment the areas with thicker haze and thinner haze respectively, and segment them into mn blocks. During the segmentation process, continuously update the preset threshold; perform further segmentation on the basis of segmentation, so that the difference in each small block is within a certain range, so as to ensure that the haze of each small block after segmentation is uniform; Step 3: Perform haze and fog processing. For haze processing, an encoder and decoder are built on U-Net. The encoder has four convolution blocks, each of which contains two 3×3 convolution filter chains, a batch normalization layer, and a ReLU activation function. A maximum pooling operation is performed after each convolution block. The decoder includes four amplification blocks, each of which includes a pixel shuffle layer, followed by a 3×3 convolution filter, batch normalization, and ReLU activation. The result of the decoder is passed to a convolution layer with a filter size of 1×1 to obtain a clear image. Dense fog processing uses a weighted average image fusion strategy to adaptively improve the accuracy of transmission map estimation; Assign a larger weight to the initial transmission map and a lower weight to the gamma-corrected version; fuse the initial transmission map with the transmission map obtained through DCP and gamma correction to adaptively improve the accuracy of transmission map estimation; then perform guided filtering on the fused transmission map of each image block; finally, obtain the defogged image block through the atmospheric scattering model I(X)=J(X)T(X)+A(1-T(X)); in the formula, I(X) represents the foggy image, J(X) represents the fog-free image, T(X) represents the transmission map, and A represents the atmospheric light; Step 4: Histogram correction: Before stitching, all image blocks are subjected to histogram correction to ensure that the colors of the reconstructed image blocks are consistent. Step 5, image stitching; restore the disordered image blocks by comparing the similarity of histograms; extract and save the left and right edges of the block images, compare the left and right edges in pairs, perform stitching operations on the block images whose similarity exceeds the preset threshold, and obtain the left and right stitched images; extract the upper and lower edges of the left and right stitched block images, compare the upper and lower edges in pairs, and perform stitching operations on the block images whose similarity exceeds the preset threshold to obtain a fog-free image; Step 6: Eliminate the stitching seams by improving median filtering and total variation smoothing; The median filter is improved, and the grayscale correction is guided on both sides of the splicing seam by the set threshold. The method only corrects the pixels on both sides that meet the set conditions, and the pixels that do not meet the conditions remain unchanged; the grayscale difference at the splicing seam is reduced; The total variation smoothing method is used to further process the median filtered image to ensure that the edge details are not affected; finally, a dehazed image is obtained.

2. According to claim 1, a non-uniform defogging method combining LSD secondary segmentation and deep learning, Features: In step 1, the RESIDE dataset is used. The RESIDE dataset is a synthetic dataset consisting of five subsets: indoor training set, outdoor training set, synthetic target test set, real-world task-driven test set, and mixed subjective test set. The indoor training set, outdoor training set, and synthetic target test set are synthetic datasets, the real-world task-driven test set is a real dataset, and the mixed subjective test set is a dataset composed of synthetic and real blurred images.

3. According to claim 1, the uneven defogging method combining LSD secondary segmentation and deep learning, Features: In the LSD secondary segmentation of step 2, proceed as follows: Input the data set in step 1 and segment each data set; 2.1 Define the coordinates of each point. According to the color attenuation prior, the difference between brightness and saturation is proportional to the division. The larger the difference, the higher the concentration; the smaller the difference, the lower the concentration; 2.2 According to the characteristics of the image, the image is divided into m*n parts, that is, mn blocks. The center of each block is selected as the center point, with a total of mn center points. The difference c(x) between the brightness and saturation of each center point is calculated, and the average difference between the maximum and minimum values ​​of the original image is set as the initial threshold, that is: c(x)=l(x)-s(x) Wherein, l(x) represents brightness, s(x) represents saturation; h represents the preset threshold; x represents the pixel coordinate; 2.3 If the difference in the coordinates of each point of the block is greater than or equal to or less than the preset threshold, Φ(x) is expanded outward until the maximum possible range is within the preset threshold, that is: Assume that the range of each block after the first division is s; If any c(x)≥h or c(x)≤h, its range expands outward by Φ(x); 2.4 If the difference of a certain coordinate of the block is not within the preset threshold range, Φ(x) is gradually reduced until the maximum possible range is within the preset threshold range; that is: In s, if there is or Then its range shrinks inward by Φ(x); 2.5 The preset threshold will be gradually updated according to the division of each block; 2.6 Adaptively segment each block after segmentation, and the difference of each small block is less than or equal to r, so as to ensure that the haze of each small block is relatively uniform; that is: Suppose the range of each small block after the second division is s' For any s', c max (x)-c min (x)≤r; r represents the maximum range of the maximum difference and the minimum difference in each small block; 2.7 Divide each small block into thick fog block and thin fog block.

4. According to claim 1, the uneven defogging method combining LSD secondary segmentation and deep learning, Features: In the step of performing haze processing and dense fog processing in step 3, the U-Net encoder-decoder structure removes haze; first, based on the encoder-decoder architecture U-Net, the encoder uses 4 convolution blocks to represent the noisy input image into a latent space. These convolution blocks extract relevant features at different scales. Each convolution block includes 2 chains of convolution filters of size 3×3, batch normalization layer and ReLU activation function. After each convolution block, a maximum pooling operation is performed to aggregate features and increase the receptive field size of subsequent convolution blocks; to reconstruct a haze-free image, 4 magnification blocks are used, each of which includes a pixel shuffle layer, followed by a convolution filter of size 3×3, batch normalization and ReLU activation; the features obtained at each level of the encoder are connected to the features of the corresponding level on the decoder side through long jump connections; the result of the decoder is passed to a convolution layer with a filter size of 1×1 to obtain a clear image; in the case of non-uniform haze, the affected area may exceed the perception of the convolution kernel, and LSD secondary segmentation is used to evenly divide the blocks, thereby solving the problem of inaccurate haze recovery; Dense fog is processed by accurate transmission map; for the segmented dense fog, the initial transmission map is obtained by DCP Among them, t 1 (x) represents the initial transmission image, r, g, b represent the three color channels of red, green, and blue, A c represents the atmospheric light of any channel, I c (y) represents any color channel of I, Ω(x) represents the window at pixel point x, and ω is a parameter; gamma correction is introduced to preprocess the transmission map obtained by DCP. DCP means that in each local area of ​​most outdoor fog-free images, at least one color channel has a very low intensity value, thereby obtaining the initial transmission map; the transmission map formula with gamma correction is is the gamma value; the image corrected by DCP gamma and the initial transmission image are fused through an adaptive fusion strategy. The adaptive fusion strategy can maintain the details of the restored image and can adaptively and effectively fuse the image with different degrees of blur. The fusion formula is: t f (x)=t 1 (x)w f (x)+t 2 (x)(1-w f (x)); In the formula, t 1 (x) is the initial transmission image, t 2 (x) is the transmission image after gamma correction, t f (x) is the fused transmission image, w f (x) is the fusion weight; The transmission map and the gamma-corrected transmission map are filtered, namely: g 1 (x)=guidf τ,ε (t 1 (x)); g 2 (x)=guidf τ,ε (t 2 (x)); τ, ε are the parameters of the guided filter, thus we get: in k is a constant; The fused transmission image is guided filtered to obtain better visual effects and solve the halo and block artifact problems of the dehazed image, namely: t f (x)=guidf τ,ε (I gray (x),t f (x)); Among them I gray (x) is the grayscale image of the original image; Then through To achieve image defogging, I(x) in the formula represents the foggy image, J(x) represents the fog-free image, A represents the atmospheric light, and t f (x) is the transmission image after fusion.

5. According to claim 1, the uneven defogging method combining LSD secondary segmentation and deep learning, Features: In step 4, the histogram correction is performed as follows: 4.1 Convert color images to grayscale images and list the grayscale levels of each image; 4.2 Calculate the probability of each histogram, that is, the proportion of pixels of each gray level in each image to the total, and select the histogram of the image with the best effect as the specified histogram; 4.3 Calculate the cumulative histogram of each image; 4.4 Calculate the absolute value of the difference between the cumulative histogram of each histogram and the cumulative histogram of the specified histogram; 4.5 Construct a grayscale mapping table and determine the mapping relationship; 4.6 Map each image grayscale to a new grayscale.

6. According to claim 1, the uneven defogging method combining LSD secondary segmentation and deep learning, Features: In step 5 of image stitching, the stitching of images is performed as follows: Image stitching is performed by histogram similarity contrast. The higher the similarity, the greater the possibility of stitching. 5.1 Extract and save the left and right edges of the image; 5.2 stitch the images whose similarity exceeds the preset threshold to obtain the left and right stitched images; 5.3 Extract the upper and lower edges of the left and right spliced ​​image blocks; 5.4 stitching the images whose similarity exceeds the preset threshold; finally obtaining a haze-free image; If the similarity value reaches 0.95, the edges of the two images are considered similar and are stitched together.

7. According to claim 1, the uneven defogging method combining LSD secondary segmentation and deep learning, Features: In step 6, in order to eliminate the seams by improving the median filter and total variation smoothing, the grayscale mean within the range Z on both sides of the seam is counted row by row, and then the grayscale values ​​of the pixels in each row within the range of the correction width W are sorted by median to obtain the grayscale median M. i , W needs to be less than Z; where i represents the number of rows; Set a threshold value, based on experience, and take 0.02; use the pixel points within the W range and the corresponding grayscale mean S i If the absolute value of the difference is greater than or equal to the threshold, the gray value of the point is modified to the corresponding M i , if the difference is less than the threshold, no change is made; Total variation smoothing eliminates the seams; the total variation smoothing model is obtained by solving the Euler-Lagrange equation through the gradient descent method: is the diffusion coefficient. In the area where the image grayscale changes sharply, the gradient value |▽μ| is large and the diffusion coefficient is small, so the diffusion is weak in the edge area and the image details can be retained. In the area where the image grayscale changes gently, the gradient value |▽μ| is small and the diffusion coefficient is large, so the diffusion ability is strong in the flat area, which solves the problem of smooth transition edges.

Images