A multi-feature enhancement method for image dehazing

Abstract

This invention provides a multi-feature enhanced image dehazing method, comprising the following steps: acquiring pairs of foggy and fog-free images of the same scene to form a training dataset; constructing an image dehazing network model; the image dehazing network model jointly estimating transmittance and atmospheric light value as new variables, and training the constructed dehazing network model using a linear combination of pixel loss function, boundary loss function, and texture loss function; inputting a single foggy image into the trained dehazing network model to obtain a dehazed image. This invention designs an end-to-end multi-feature enhanced dehazing neural network that requires no prior knowledge. It adopts a strategy of simplifying the projection map and atmospheric light map into feature maps, decomposing the fog image into a base layer and a detail layer, weighting and fusing the weights of each image layer to estimate the transmission map to recover the fog-free image, and using a joint loss strategy to train the model, further enhancing the model's dehazing accuracy.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, and in particular relates to a multi-feature enhancement method for image dehazing. Background Technology

[0002] Environmental perception technology is a crucial component of autonomous driving, especially with the rapid advancements in image processing and computer hardware technologies in recent years, leading to the widespread application of vision-based environmental perception technologies. However, visual environmental perception technologies are highly dependent on environmental visibility conditions. Particularly in certain regions, under the influence of frequent rain, fog, and other adverse weather conditions, camera imaging suffers from low clarity and contrast, color distortion, and blurred details. Furthermore, current vision-based algorithms for object detection and semantic segmentation are not ideal for images in rainy or foggy conditions. Traffic information is obscured by varying degrees of fog, making it easy for visual sensors to miss detections, misidentify locations, and make incorrect identifications, potentially leading to significant safety hazards during driving. Therefore, image restoration technology for foggy and other adverse environments is one of the challenges faced by environmental perception technologies in harsh conditions. [1] .

[0003] Hazy images are a typical image degradation problem. In recent years, most image dehazing methods developed by scholars both domestically and internationally are based on atmospheric scattering models, the core of which is calculating the atmospheric light value and transmittance map of the image. However, the estimation methods for these two parameters vary. Among them, the most representative is the DCP (DarkChanel Prior) algorithm. [2] The method uses the idea of ​​dark channel prior to guide the estimation of atmospheric light value and transmission map. Its advantage is that the defogging effect is obvious, but there is a problem of color distortion in the sky area. Reference [3] proposed a method to decompose the simplified dark channel of foggy image into base layer and detail layer using weighted guided image filtering, and to estimate the projection map on the base layer. The performance of this method is limited, and the improvement of details in feature-dense areas is not obvious. Reference [4] uses color attenuation prior information to construct the depth map of the image using the brightness and saturation of the image, and then estimates the atmospheric light value and projection map, but there is a problem that the depth estimation takes a long time. Reference [5] uses mathematical induction of RGB color space to cluster the foggy areas in the image. This method has good detail recovery ability, but it will fail in scenes with bright atmospheric light. Reference [6] uses a global image filtering method to estimate the projection map using a combination of global edges and structures, and has good detail recovery ability. Reference [7] proposed an atmospheric light model for reconstructing fog-free images, and uses the brightness, saturation and gradient information of the image to calculate the projection map. Since the image degradation problem is not considered, the resulting fog-free image lacks realism.

[0004] Furthermore, with the continuous development of computer technology, image dehazing algorithms based on deep learning have also made significant progress. Currently, there are two main approaches to dehazing algorithms based on this method: one is to use a convolutional neural network (CNN) to generate certain parameters of an atmospheric scattering model, and then use the atmospheric scattering model to recover the image. [8-10] Another approach is to use a Generative Adversarial Network (GAN) to directly generate a clear, fog-free image from a blurred image. [11-14] Reference [8] first proposed a dehazing network called DehazeNet, which estimates the transmittance of the foggy image through the deep architecture of CNN and then reconstructs the fog-free image by substituting it into the atmospheric scattering model. Reference

[12] proposed a dehazing network using superimposed conditional GAN, which can independently recover each color channel of RGB and has good generalization. In summary, the dehazing method based on neural networks has the characteristics of high efficiency, but due to the lack of real training data or prior parameters, its dehazing performance is limited to a certain extent.

[0005] In summary, current dehazing methods only focus on enhancing the features of the image itself, such as pixel and color information, while ignoring graphic information. Furthermore, existing object detection and semantic segmentation algorithms are mostly based on information such as image texture boundaries for detection. Therefore, the improvement in performance for reprocessing foggy images enhanced by traditional algorithms is not particularly significant. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention provides a fast and high-quality multi-feature enhancement method for image dehazing.

[0007] The technical solution adopted by this invention to solve its technical problem is: a multi-feature enhancement image dehazing method, comprising the following steps:

[0008] S1. Obtain pairs of foggy and fog-free images of the same scene to form a training dataset;

[0009] S2. Construct an image dehazing network model;

[0010] S3. The image dehazing network model jointly estimates the transmittance t(x) and atmospheric light value A as variable K(x), and trains the constructed dehazing network model using a linear combination of pixel loss function, boundary loss function and texture loss function;

[0011] S4. Input a single foggy image into the trained dehazing network model to obtain a dehazed image;

[0012] The image dehazing network model consists of three learning blocks: low, medium, and high. Each learning block is composed of a channel segmentation layer, a first tensor concatenation layer, a first convolutional layer, a first ReLU activation layer, a second tensor concatenation layer, a second convolutional layer, a second ReLU activation layer, a third tensor concatenation layer, a third convolutional layer, and a third ReLU activation layer, all connected in a series.

[0013] The first tensor splicing layer merges multiple channel tensors from the output of the channel splitting layer; the second tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output tensor of the second ReLU activation layer; and the third tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output tensor of the third ReLU activation layer.

[0014] The outputs of the first, second, and third ReLU activation layers are merged by the fourth tensor concatenation layer and used as the input of the feature extraction unit (CMT). The output of the CMT in the low-level learning block is used as the input of the channel segmentation layer in the intermediate learning block, and the output of the CMT in the intermediate learning block is used as the input of the channel segmentation layer in the high-level learning block.

[0015] Meanwhile, the outputs of the third ReLU activation layer and the feature extraction unit CMT in the low, medium and high learning blocks are merged by the fifth tensor concatenation layer and then connected in series with a channel attention mechanism and a convolutional layer to finally output a dehazed image.

[0016] As a further preferred embodiment, the feature extraction unit (CMT) is a CNN-based Transformer structure, which adopts a QKV structure. The K stage includes a series of 3×3 convolutional layers, a normalization layer, and a ReLU activation layer; the Q stage includes a series of 1×1 convolutional layers, a normalization layer, a ReLU activation layer, and a 1×1 convolutional layer. The input of the CMT and the output of the K stage are merged into a tensor by a feature fusion layer and then passed to the Q stage. The output of the Q stage is activated by a Softmax layer; the V stage includes a series of 1×1 convolutional layers and a normalization layer; the output of the V stage is multiplied by the output of the Q stage (activated by the Softmax layer) and then merged with the output of the K stage as a tensor to obtain the final output of the feature extraction unit (CMT).

[0017] As a further preferred embodiment, the channel attention mechanism includes a serially connected pooling layer, a first fully connected layer, a ReLU activation layer, a second fully connected layer, and a sigmoid layer. The output of the sigmoid layer is multiplied by the input of the channel attention mechanism to obtain the final output of the channel attention mechanism.

[0018] As a further preferred implementation, the expression for variable K(x) is:

[0019]

[0020] Where x represents the pixel of the image, I(x) represents the foggy image, A represents the atmospheric light value, t(x) represents the transmittance, and b is a bias constant with a default value of 1.

[0021] As a further preferred embodiment, the joint loss function L after linear combination in step S3 is:

[0022] L = 0.8 × L MAE +0.1×L boundary +0.1×L texture

[0023] Among them, L MAE Let L be the pixel loss function. boundary Let L be the boundary loss function. texture This is the texture loss function.

[0024] As a further preferred embodiment, the expression for the pixel loss function is:

[0025]

[0026] Where X is a real labeled image (foggy image) in the training dataset, Y is the fog-free image obtained after processing the corresponding foggy image by the MFED-Net model, and m and n are the width and height of X, respectively.

[0027] As a further preferred embodiment, the expression for the boundary loss function is:

[0028]

[0029] in,

[0030]

[0031] M is the mask, and s is the number of non-zero elements in M.

[0032] As a further preferred embodiment, the expression for the texture loss function is:

[0033]

[0034] Among them, the weight mask W Laplacian =F Laplacian / 255, F Laplacian This is the original texture feature map corresponding to image X in the RGB color channels.

[0035] The positive effects of this invention: This patent addresses the texture and boundary information in foggy road images by designing an end-to-end multi-feature enhanced dehazing neural network (MFED-Net) based on ASM theory. It requires no prior knowledge and employs a strategy that simplifies the projection map and atmospheric light map into feature maps. The foggy image is decomposed into a base layer and a detail layer, with weights assigned to each image layer and then fused to estimate the transport map and recover the fog-free image. Furthermore, a joint loss strategy targeting pixels, texture, and boundaries is used to train the model, further enhancing its dehazing accuracy. Experimental results on different types of images, including outdoor synthetic fog images, indoor synthetic fog images, and real traffic fog images, validate the performance of the proposed dehazing method. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of the image dehazing network model MFED-Net described in this invention;

[0037] Figure 2 This is a schematic diagram of the feature extraction unit CMT described in this invention;

[0038] Figure 3 This is a schematic diagram of the attention channel mechanism described in this invention;

[0039] Figure 4 This is a comparison chart of the original fog map with the defogging method of this invention and other existing defogging methods (based on the defogging effect of the Cityscapes autonomous driving dataset). Figure 4 Column (a) in the image corresponds to the original fog map. Figure 4 Column (b) in the diagram corresponds to the effect of the existing GridDehazeNet method. Figure 4 Column (c) in the diagram corresponds to the effect of the existing FFA method. Figure 4 Column (d) in the diagram corresponds to the effect of the existing PCFAN method. Figure 4 Column (e) in the diagram corresponds to the effect of the existing YOLY method. Figure 4 Column (f) in the diagram corresponds to the effect of the existing MSBDN method. Figure 4 Column (g) in the diagram corresponds to the effect of the existing LD method. Figure 4 Column (h) in the diagram corresponds to the effect diagram of the method in the embodiment of the present invention. Figure 4 Column (i) in the image corresponds to the clear image corresponding to the original fog map.

[0040] Figure 5 This is a comparison chart of the effects of the original fog image, the defogging method of this invention, and other existing defogging methods (based on the defogging effect of the RTTS natural fog dataset). Figure 5 Column (a) in the image corresponds to the original fog map. Figure 5Column (b) in the diagram corresponds to the effect of the existing GridDehazeNet method. Figure 5 Column (c) in the diagram corresponds to the effect of the existing FFA method. Figure 5 Column (d) in the diagram corresponds to the effect of the existing PCFAN method. Figure 5 Column (e) in the diagram corresponds to the effect of the existing YOLY method. Figure 5 Column (f) in the diagram corresponds to the effect of the existing MSBDN method. Figure 5 Column (g) in the diagram corresponds to the effect of the existing LD method. Figure 5 The (h) column in the figure corresponds to the effect diagram of the method in the embodiment of the present invention. Detailed Implementation

[0041] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0042] Many image dehazing algorithms, whether end-to-end deep learning algorithms or dark channel prior algorithms, are based on the atmospheric light scattering model (ASM), which has been used as the benchmark for image dehazing. The mathematical description of the ASM model for single-image dehazing is as follows:

[0043] I(x)=J(x)t(x)+A[1-t(x)](1)

[0044] Where x represents the number of pixels in the image, I(x) represents the hazy image, J(x) represents the dehazed image, A represents the atmospheric light value, and t(x) represents the transmittance; J(x)t(x) is the direct attenuation term, and A[1-t(x)] represents the intensity effect of natural light under the influence of the air medium. Dividing both sides of the equation by t(x) and rearranging, we get:

[0045]

[0046] Most existing deep learning-based methods estimate t(x) and A separately and substitute them into equation (2) to obtain a fog-free image. However, this method of simultaneously estimating atmospheric light value and transmittance is very time-consuming and has a large cumulative error. Therefore, this invention adopts a transformation method to jointly estimate t(x) and A into a new variable K(x), as shown in equation (3):

[0047] J(x)=K(x)×I(x)-K(x)+b(3)

[0048] Where b is a bias constant with a default value of 1, and the relationship between K(x), t(x), and A is shown in equation (4):

[0049]

[0050] Based on the above algorithm principle, referring to Figure 1-3This invention provides a multi-feature enhanced image dehazing method, comprising the following steps:

[0051] S1. Obtain pairs of foggy and fog-free images of the same scene to form a training dataset;

[0052] S2. Construct an image dehazing network model (Multi-Feature Enhancement Dehazing Network);

[0053] S3. As mentioned above, this patent employs a joint estimation method, using a dehazing network design strategy that jointly estimates the transmittance t(x) and atmospheric light value A as a single parameter. This approach reduces image computation and the number of network parameters, thereby saving computational costs and accelerating dehazing. Furthermore, it improves model accuracy from an error perspective. In summary, the image dehazing network model jointly estimates transmittance t(x) and atmospheric light value A as a variable K(x), and trains the constructed dehazing network model using a linear combination of pixel loss function, boundary loss function, and texture loss function.

[0054] S4. Input a single foggy image into the trained dehazing network model to obtain a dehazed image.

[0055] Previous dehazing neural networks were mostly based on CNN structures. While these networks do perform well in detail restoration, they also have drawbacks in overall image perception, such as color distortion or ghosting, due to the small convolutional kernels of CNNs. Of course, increasing the number of CNN convolutional kernels or using Transformers to replace some convolutions can improve image quality, but the resulting parameter increase and tensor computation cost are unacceptable. Therefore, this patent seeks a neural network structure that combines the global perception capability of Transformers with the local perception capability and computational speed of CNNs. Thus, this patent proposes a CNN-based Transformer structure, CMT (Convolution Mapping Transformer), as follows... Figure 2 As shown, similar to the classic Transformer, the CMT block in this patent also adopts a QKV structure. The difference lies in that the feature extraction stage in this patent uses convolutions throughout. Furthermore, except for the K-stage which uses a 3×3 convolution, all other convolutions use only 1×1 kernels to change the channel depth. This approach ensures the same extraction effect as the Transformer while maintaining the fast computation speed of a CNN. Specifically, the feature extraction unit CMT is a CNN-based Transformer structure (e.g., ...). Figure 2 As shown, it adopts a QKV structure, where the K stage includes a series of 3×3 convolutional layers, a normalization layer, and a ReLU activation layer; the Q stage includes a series of 1×1 convolutional layers, a normalization layer, a ReLU activation layer, and a 1×1 convolutional layer. The input of the CMT and the output of the K stage are tensor-merged and then passed to the Q stage through a feature fusion layer. The output of the Q stage is activated by a Softmax layer; the V stage includes a series of 1×1 convolutional layers and a normalization layer; the output of the V stage is matrix-multiplied with the output of the Q stage after activation by the Softmax layer, and then tensor-merged with the output of the K stage to serve as the final output of the feature extraction unit (CMT).

[0056] Based on the aforementioned feature extraction unit (CMT), the overall structural design of the dehazing network model in this embodiment is as follows: Figure 1 As shown, the image dehazing network model includes three learning blocks: low, medium, and high. Each learning block consists of a channel segmentation layer, a first tensor concatenation layer, a first convolutional layer, a first ReLU activation layer, a second tensor concatenation layer, a second convolutional layer, a second ReLU activation layer, a third tensor concatenation layer, a third convolutional layer, and a third ReLU activation layer, all connected in a series.

[0057] The first tensor splicing layer merges multiple channel tensors from the output of the channel splitting layer; the second tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output tensor of the second ReLU activation layer; and the third tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output tensor of the third ReLU activation layer.

[0058] The outputs of the first, second, and third ReLU activation layers are merged by the fourth tensor concatenation layer and used as the input of the feature extraction unit (CMT). The output of the CMT in the low-level learning block is used as the input of the channel segmentation layer in the intermediate learning block, and the output of the CMT in the intermediate learning block is used as the input of the channel segmentation layer in the high-level learning block.

[0059] Meanwhile, the outputs of the third ReLU activation layer and the feature extraction unit CMT in the low, medium and high learning blocks are merged by the fifth tensor concatenation layer and then connected in series with a channel attention mechanism and a convolutional layer to finally output a dehazed image.

[0060] Among them, such as Figure 3 As shown, the channel attention mechanism includes a serially connected pooling layer, a first fully connected layer, a ReLU activation layer, a second fully connected layer, and a sigmoid layer. The output of the sigmoid layer is multiplied by the input of the channel attention mechanism to obtain the final output of the channel attention mechanism.

[0061] In summary, the dehazing network model described in this embodiment adopts a residual connection form, that is, CNN and CMT are integrated, while retaining some of its own features to be passed to the next layer of the network connected to it. This form is named MFED-Net (Multi-feature Enhancement Dehaze Network). Figure 1 As shown, MFED-Net contains three blocks for learning low, medium, and high-level image semantic features. For example, for block 1, the 12-channel tensor after channel addition processing by Conv0 is split into 4×3 channels by Split1 (channel splitting layer). Then, the first two 3-channel tensors are merged by Concat1 (cat1, the first tensor concatenation layer), followed by convolution processing by Conv1 (the first convolutional layer). Then, it is merged with the third 3-channel tensor by Concat2 (cat2, the second tensor concatenation layer), followed by convolution processing by Conv2 (the second convolutional layer). This process is repeated to complete the operation of block 1, resulting in the feature map of the primary learning block Block 1 containing global and local features. Then, similar operations are performed on blocks 2 and 3. At this point, multiple tensors containing semantic features are obtained. Because CNNs lose low-level semantics learned by earlier layers during high-level representation, this patent adopts a form of residual connection between low (Conv3, CMT1), medium (Conv6, CMT2), and high (Conv9, CMT3) learning layers. That is, each layer participates in the pre-output processing of the network [the network output contains features from multiple layers and stages, such as concat13 (the fifth tensor concatenation layer), which is a fusion of features from multiple layers (Conv3, Conv6, Conv9, CMT1, CMT2, CMT3) and multiple stages (Block1 low-level, Block2 medium-level, Block3 high-level)], to reduce the loss of semantic information and obtain feature maps containing different features from each learning stage. Finally, after a process such as Figure 3 The Channel Attention (CA) mechanism shown performs channel weighting on the 36-channel feature map, followed by a Conv10 channel reduction process, which restores the image color channels without losing features.

[0062] The network structure parameters are shown in Table 1:

[0063] Table 1 Detailed parameters of MEFD-Net

[0064]

[0065] The specific algorithm flow and the parameters used are shown in Tables 2 and 3:

[0066] Table 2. MFED-Net Algorithm Flowchart

[0067]

[0068]

[0069] Table 3 Parameter Description

[0070]

[0071]

[0072] Regarding the loss function

[0073] Existing algorithms primarily use the L2 loss function to optimize the entire network. However, this approach only focuses on the pixels themselves and fails to consider the differences between the dehazed and labeled images at a more abstract perceptual level, such as edge and texture features. Therefore, a joint loss function that incorporates pixel, edge, and texture information is needed.

[0074] Let X be a real labeled image in the training set, and Y be the dehazed image of the corresponding hazy image after inference by MFE-Net. Next, loss functions are designed for the pixels, edges and textures of X and Y respectively.

[0075] (1) Pixel loss

[0076] In actual training, most predicted values ​​outside the dense fog obscure the target value, with small errors and variances. However, the presence of outliers in Y significantly increases the error. Since the mean squared error (MSE) results in a much larger loss at points with large errors than the mean absolute error (MAE), it assigns greater weight to outliers. The model then tries to reduce the error caused by outliers, leading to a decline in overall model performance. Therefore, MAE is more effective when the training data contains a large number of outliers. Thus, the pixel loss adopts a strategy of calculating MAE, specifically defined as follows:

[0077]

[0078] Where m and n are the width and height of X, respectively.

[0079] (2) Boundary loss

[0080] First, the Canny operator with a threshold of (100, 200) is used to extract features such as boundaries and textures from the label image X, obtaining the corresponding mask M. Then, M is summed over X and Y respectively to obtain the edge image X. mask Y mask As shown in the following formula:

[0081]

[0082] Then regarding X mask and Y mask The MAE calculation is performed as shown in the following formula:

[0083]

[0084] Where s is the number of non-zero elements in M.

[0085] (3) Texture loss

[0086] Texture information recovery is one of the key features of MFE dehazing neural networks, making the design of a reasonable texture loss function essential. First, the Laplacian operator is used to obtain the original texture feature map F corresponding to the label image X in the RGB color channels. Laplacian and F Laplacian The grayscale values ​​are normalized to obtain the weight mask W. Laplacian As shown in Equation 8:

[0087] W Laplacian =F Laplacian / 255 (8)

[0088] Performing MAE and weighted summation operations on X and Y, we get:

[0089]

[0090] In summary, the joint loss function is expressed as follows:

[0091] L = 0.8 × L MAE +0.1×L boundary +0.1×L texture

[0092] To evaluate the defogging capability of the defogging method described in this invention, this patent employs an experimental strategy under different synthetic fog concentrations and natural fog conditions. Specifically, 150 images were randomly selected from the CITYSCAPES autonomous driving dataset, and three fog images with different concentrations were synthesized at a concentration coefficient parameter β = 0.005–0.02, totaling 500 images as the test set. The resulting comparison results are as follows: Figure 4 As shown; 500 images were randomly selected from the RTTS dataset for real-world foggy weather as the outdoor real-world foggy weather test set, and the comparison results are as follows. Figure 5 As shown.

[0093] Figure 4 , Figure 5This paper visually demonstrates the performance of MFED-Net and existing methods such as SOTA on the Cityscapes and RTTS datasets. Almost all existing methods exhibit significant color distortion under high fog density. When the fog density distribution is uneven, methods such as MSBDN and YOLY fail to completely remove fog from the distant parts of the image, LD has excessively low contrast in the near-field parts, and GridDehazeNet exhibits artifacts. In contrast, MFED-Net restores image colors to the greatest extent and retains more image details on both synthetic and natural datasets.

[0094] Table 4 shows the SSIM and PSNR data of each method on the Cityscapes dataset. The best and second-best data are shown in bold and underlined fonts, respectively. It can be seen that MFED-Net performs well on both evaluation metrics and is closer to a clear image.

[0095] Table 4 shows the SSIM and PSNR data for each method on the Cityscapes dataset.

[0096]

[0097] In summary, this invention proposes a multi-feature enhanced dehazing neural network for the characteristics of traffic images in foggy weather. It utilizes CNN and CNN-mapped transformer modules, combining the detail extraction capability of traditional CNN with the global receptive field capability of transformer. Furthermore, the number of model parameters is not significantly increased compared to the pure CNN network structure, and the speed loss is minimal to a certain extent.

[0098] References

[0099] [1]Badue C,Guidolini R,Carneiro RV,et al.Self-driving cars:A survey[J].Expert Systems with Applications,2021,165:113816.

[0100] [2]He K,Sun J,Tang X.Single image haze removal using dark channel prior[J].IEEE transactions on pattern analysis and machine intelligence,2010,33(12):2341-2353.

[0101] [3]Li Z,Zheng J.Edge-preserving decomposition-based single image hazeremoval[J].IEEE Transactions on Image Processing,2015,24(12):5432-5441.

[0102] [4]Li Z,Zheng J,Zhu Z,et al.Weighted guided image filtering[J].IEEETransactions on Image processing,2014,24(1):120-129.

[0103] [5]Berman D,Avidan S.Non-local image dehazing[C] / / Proceedings of theIEEE conference on computer vision and pattern recognition.2016:1674-1682.

[0104] [6]Li,Zhengguo,and Jinghong Zheng."Single image de-hazing usingglobally guided image filtering."IEEE Transactions on Image Processing 27.1(2017):442-450.

[0105] [7]Ju,Mingye,Zhenfei Gu,and Dengyin Zhang."Single image haze removalbased on the improved atmospheric scattering model."Neurocomputing 260(2017):180-191.

[0106] [8]Cai B,Xu X,Jia K,et al.Dehazenet:An end-to-end system for singleimage haze removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-5198.

[0107] [9]LI B,PENG X,WANG Z,et al.An all-in-one network for dehazing andbeyond[J].arXiv preprint arXiv:1707.06543,2017.

[0108]

[10] LIU X,MA Y,SHI Z,et al.Griddehazenet:Attention-based multi-scalenetwork for image dehazing[C] / / Proceedings of the IEEE / CVF InternationalConference on Computer Vision.2019:7314-7323.

[0109]

[11] LEE B U,LEE K,OH J,et al.CNN-Based Simultaneous Dehazing andDepth Estimation[C] / / 2020 IEEE International Conference on Robotics andAutomation(ICRA).IEEE,2020:9722-9728.

[0110]

[12] P L,SAPPA A D,VINTIMILLA B X,et al.Deep learning basedsingle image dehazing[C] / / Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition Workshops.2018:1169-1176.

[0111]

[13] ENGIN D, A,KEMAL EKENEL H.Cycle-dehaze:Enhanced cyclegan forsingle image dehazing[C] / / Proceedings of the IEEE conference on computervision and pattern recognition workshops.2018:825-833.

[0112]

[14] CHEN D,HE M,FAN Q,et al.Gated context aggregation network forimage dehazing and deraining[C] / / 2019 IEEE winter conference on applicationsof computer vision(WACV).IEEE,2019:1375-1383.

[0113]

[15] Zhang H,Patel V M.Densely connected pyramid dehazing network[C] / / Proceedings of the IEEE conference on computer vision and patternrecognition.2018:3194-3203.

[0114]

[16] Qin X,Wang Z,Bai Y,et al.FFA-Net:Feature fusion attention networkfor single image dehazing[C] / / Proceedings of the AAAI Conference onArtificial Intelligence.2020,34(07):11908-11915.

[0115]

[17] Zhang X,Wang T,Wang J,et al.Pyramid channel-based featureattention network for image dehazing[J].Computer Vision and ImageUnderstanding,2020,197:103003.

[0116]

[18] Dong H,Pan J,Xiang L,et al.Multi-scale boosted dehazing networkwith dense feature fusion[C] / / Proceedings of the IEEE / CVF conference oncomputer vision and pattern recognition.2020:2157-2167.

[0117]

[19] Li B,Gou Y,Gu S,et al.You only look yourself:Unsupervised and untrained single image dehazing neural network[J].International Journal ofComputer Vision,2021,129(5):1754-1767.

[0118]

[20] Ullah H, Muhammad K, Irfan M, et al.Light-DehazeNet: a novellightweight CNN architecture for single image dehazing[J]. IEEE Transactionson Image Processing, 2021,30:8968-8982.

[0119] The above description is only a preferred embodiment of the present invention. It should be understood that the above description of the embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, etc. made within the idea and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-feature enhancement image dehazing method, characterized in that, Includes the following steps: S1. Obtain pairs of foggy and fog-free images of the same scene to form a training dataset; S2. Construct an image dehazing network model; S3. The image dehazing network model jointly estimates the transmittance t(x) and atmospheric light value A as variable K(x), and trains the constructed dehazing network model using a linear combination of pixel loss function, boundary loss function and texture loss function; The expression for variable K(x) is: Where x represents the pixels of the image, I(x) represents the foggy image, A represents the atmospheric light value, t(x) represents the transmittance, and b is a bias constant with a default value of 1; S4. Input a single foggy image into the trained dehazing network model to obtain a dehazed image; The image dehazing network model consists of three learning blocks: low, medium, and high. Each learning block is composed of a channel segmentation layer, a first tensor concatenation layer, a first convolutional layer, a first ReLU activation layer, a second tensor concatenation layer, a second convolutional layer, a second ReLU activation layer, a third tensor concatenation layer, a third convolutional layer, and a third ReLU activation layer, all connected in a series. The first tensor splicing layer merges multiple channel tensors from the output of the channel splitting layer; the second tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output of the second ReLU activation layer; and the third tensor splicing layer merges a single channel tensor from the output of the channel splitting layer with the output of the third ReLU activation layer. The outputs of the first, second, and third ReLU activation layers are merged by the fourth tensor concatenation layer and used as the input of the feature extraction unit (CMT). The output of the CMT in the low-level learning block is used as the input of the channel segmentation layer in the intermediate learning block, and the output of the CMT in the intermediate learning block is used as the input of the channel segmentation layer in the high-level learning block. Meanwhile, the outputs of the third ReLU activation layer and the feature extraction unit CMT in the low, medium and high learning blocks are merged by the fifth tensor concatenation layer and then connected in series with a channel attention mechanism and a convolutional layer to finally output a dehazed image.

2. The image dehazing method with multi-feature enhancement according to claim 1, characterized in that: The feature extraction unit (CMT) is a CNN-based Transformer structure, employing a QKV architecture. The K stage comprises a series of 3×3 convolutional layers, a normalization layer, and a ReLU activation layer. The Q stage comprises a series of 1×1 convolutional layers, a normalization layer, a ReLU activation layer, and a 1×1 convolutional layer. A feature fusion layer merges the CMT input with the K stage output as a tensor before passing it to the Q stage. The Q stage output is activated by a Softmax layer. The V stage comprises a series of 1×1 convolutional layers and a normalization layer. The V stage output is multiplied by the Q stage output (activated by the Softmax layer) and then merged with the K stage output as a tensor to obtain the final output of the CMT.

3. The image dehazing method with multi-feature enhancement according to claim 1, characterized in that: The channel attention mechanism includes a serially connected pooling layer, a first fully connected layer, a ReLU activation layer, a second fully connected layer, and a sigmoid layer. The output of the sigmoid layer is multiplied by the input of the channel attention mechanism to obtain the final output of the channel attention mechanism.

4. The image dehazing method with multi-feature enhancement according to claim 1, characterized in that: The joint loss function L after linear combination in step S3 is: L=0.8×L MAE +0.1×L boundary +0.1×L texture Among them, L MAE Let L be the pixel loss function. boundary Let L be the boundary loss function. texture This is the texture loss function.

5. The image dehazing method with multi-feature enhancement according to claim 4, characterized in that: The expression for the pixel loss function is: Where X is a real labeled image (foggy image) in the training dataset, Y is the fog-free image obtained after processing the corresponding foggy image by the MFE-Net model, and m and n are the width and height of X, respectively.

6. The image dehazing method with multi-feature enhancement according to claim 4, characterized in that: The expression for the boundary loss function is: in, M is the mask, and s is the number of non-zero elements in M.

7. The image dehazing method with multi-feature enhancement according to claim 4, characterized in that: The expression for the texture loss function is: Among them, the weight mask W Laplacian =F Laplacian / 255, F Laplacian This is the original texture feature map corresponding to image X in the RGB color channels.

Images