Offshore image defogging method based on multi-branch pyramid large kernel convolutional network

Abstract

The application provides a marine image defogging method based on a multi-branch pyramid large kernel convolutional network, comprising: acquiring a to-be-processed image; inputting the to-be-processed image into a pre-trained multi-branch pyramid large kernel encoding-decoding network to obtain a defogged image; wherein the multi-branch pyramid large kernel encoding-decoding network comprises a first module, a second module, a multi-branch pyramid module, a self-attention mechanism module and a global path aggregation module. The application solves the marine image defogging and clarification task by constructing a multi-branch feature pyramid architecture to form a hybrid representation space of reflection prediction, coloring prediction and semantic prediction, and simultaneously introduces a reparameterization method to form a combined model stacked by a large number of different receptive fields and further makes the combined receptive field more abundant.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a sea image defogging method based on a multi-branch pyramid large kernel convolutional network. BACKGROUND

[0002] At present, with the continuous increase of human-computer combination, the battlefield force organization mode of future war will undergo a revolutionary change. Unmanned aerial vehicles, unmanned underwater vehicles, combat robots and other new intelligent weapons will play an important role in future battlefield and undertake tasks such as battlefield reconnaissance, monitoring, attack and communication relay. Real-time monitoring of sea scene is the basis for sea battlefield situation assessment and threat estimation. However, sea scene images are usually accompanied by fog, clouds and other interference, which reduces the effectiveness of image content and blurs image information, which will have a serious impact on many downstream tasks.

[0003] Real-time monitoring of sea scene is the basis for sea battlefield situation assessment and threat estimation. The core task of real-time monitoring of optical remote sensing image sea surface target is how to obtain real-time high-definition sea surface scene image, which has broad application prospects in fishery management, sea rescue, sea traffic monitoring and sea battlefield situation awareness. However, due to the large amount of aerial image data and the influence of sea surface conditions, weather conditions, lighting conditions and imaging detector parameters, there are natural environmental disturbances such as fog, sea clutter and clouds on the sea surface, which will have a negative impact on visual-based decision-making in real-time systems. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application provides a sea image defogging method based on a multi-branch pyramid large kernel convolutional network, comprising:

[0005] obtaining a to-be-processed image;

[0006] inputting the to-be-processed image into a pre-trained multi-branch pyramid large kernel encoding-decoding network to obtain a defogging image;

[0007] The multi-branch pyramid large kernel encoding-decoding network comprises a first module, a second module, a multi-branch pyramid module, a self-attention mechanism module and a global path aggregation module.

[0008] Preferably, the first module is used for depth convolution of the to-be-processed image, comprising a first convolution layer, a second convolution layer, a third convolution layer and a fourth convolution layer connected in sequence; the number of channels in the first module is 64.

[0009] Preferably, the first convolution layer is a 3x3 convolution layer;

[0010] The second convolution layer is a 7x7 convolution layer;

[0011] the third convolutional layer is a 1x1 convolutional layer;

[0012] the fourth convolutional layer is a 7x7 convolutional layer.

[0013] Preferably, the second module comprises a first processing unit and a second processing unit; the output end of the first processing unit and the input end of the second processing unit are pixel-by-pixel added; the first processing unit comprises a batch normalization, a first 1x1 convolutional layer, a second 13x13 convolutional layer and a third 1x1 convolutional layer connected in sequence; the second processing unit comprises a batch normalization, a fourth 1x1 convolutional layer, a Swish activation function and a fifth 1x1 convolutional layer connected in sequence.

[0014] Preferably, the multi-branch pyramid module comprises a first branch and a second branch;

[0015] the first branch comprises a first ladder unit, a first conversion unit, a second ladder unit, a second conversion unit, a third ladder unit, a fourth ladder unit, a first up-sampling unit, a fifth ladder unit, a second up-sampling unit and a sixth ladder unit connected in sequence;

[0016] the second branch comprises a first ladder unit, a first conversion unit, a second ladder unit, a second conversion unit, a third ladder unit, a third conversion unit, a fourth ladder unit, a fifth ladder unit, a first up-sampling unit, a sixth ladder unit, a second up-sampling unit, a seventh ladder unit, a third up-sampling unit and an eighth ladder unit connected in sequence;

[0017] Preferably, the self-attention mechanism module comprises a first global average pooling layer, a first full connection layer and a second full connection layer; the global average pooling layer, the first full connection layer and the second full connection layer are connected in sequence to generate an attention score of the image to be processed, and a feature containing attention is obtained based on the attention score.

[0018] Preferably, the global path aggregation module comprises a second global average pooling layer, a third full connection layer, a fourth full connection layer and an up-sampling layer connected in sequence to obtain a global feature; the global feature and the feature containing attention are pixel-by-pixel aggregated to obtain a local-global hybrid feature;

[0019] the local-global hybrid feature is processed by a global 3x3 convolutional layer and a global up-sampling to obtain a defogged image;

[0020] the defogging level of the defogged image is determined by a combination loss function, and when the defogging level is greater than a preset threshold, it is determined that the defogged image meets the requirements.

[0021] Preferably, the calculation formula corresponding to the combination loss function is as follows:

[0022] L total = α1L adv + α2L MSE + α3L per

[0023] wherein, the L adv represents an adversarial loss function expression; the L MSE represents an MSE loss function expression; and the L per represents a perceptual similarity loss function expression; α1 represents a weight configuration parameter occupied by the adversarial loss; α2 represents a weight configuration parameter occupied by the MSE loss; and α3 represents a weight configuration parameter occupied by the perceptual similarity loss.

[0024] Preferably, the adversarial loss function expression L adv is as follows:

[0025]

[0026] wherein, B represents the number of samples in a small batch, i = 1, 2, …, B; D() represents the output of the discriminator; G() represents the generated image; and z represents the input foggy image;

[0027] The MSE loss function expression L MSE is as follows:

[0028]

[0029] wherein, K represents the number of pixels in the generated image, j = 1, 2, …, K, and R represents the real fog-free image;

[0030] The perceptual similarity loss function expression L per is as follows:

[0031]

[0032] wherein, P represents the number of elements in the output feature map of the Conv3-3 layer in the VGG 16 model, n = 1, 2, …, P.

[0033] Preferably, before the inputting of the to-be-processed image into the pre-trained large-core encoding-decoding network of the multi-branch pyramid, the method further comprises:

[0034] performing one-stage processing on the to-be-processed image by a dark channel prior method to obtain fog depth estimation sub-information of the to-be-processed image;

[0035] performing image fusion on the fog depth estimation sub-information of the to-be-processed image and the to-be-processed image to obtain a one-stage input image; ​

[0036] input the primary input image into a pre-trained large kernel encoding-decoding network of a multi-branch pyramid.

[0037] Compared with the closest prior art, the present application has the following beneficial effects:

[0038] The present application provides a sea image defogging method based on a multi-branch pyramid large kernel convolutional network, comprising: obtaining a to-be-processed image; inputting the to-be-processed image into a pre-trained large kernel encoding-decoding network of a multi-branch pyramid to obtain a defogging image; wherein the large kernel encoding-decoding network of the multi-branch pyramid comprises a first module, a second module, a multi-branch pyramid module, a self-attention mechanism module and a global path aggregation module; the present application constructs a large kernel encoding-decoding network of a multi-branch pyramid to form a hybrid representation space of reflection prediction, colorization prediction and semantic prediction, and deep convolution is applied in the process to reduce the amount of calculation, and large kernel convolution is adopted when deep convolution is performed, which is conducive to providing sufficient receptive field and spatial information and obtaining more nonlinear and cross-channel information communication; three branches are included in the multi-branch pyramid module to perform reflection prediction, colorization prediction and semantic prediction, respectively; the reflection and colorization prediction tasks provide rich color and texture complementary information for the no-fog image generation task, so that the network can generate a high-quality no-fog image with natural color and fine details.

[0039] Other features of the present application will be set forth in the following description of the application, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0040] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings are included to provide a further understanding of the present application and are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and serve to explain the present application, and do not constitute a limitation of the present application. In the drawings:

[0042] Figure 1 A flowchart of a sea image defogging method based on a multi-branch pyramid large kernel convolutional network provided by the present application;

[0043] Figure 2 A system architecture diagram of a sea image defogging method based on a multi-branch pyramid large kernel convolutional network provided by the present application;

[0044] Figure 3 A deep convolution process diagram in a sea image defogging method based on a multi-branch pyramid large kernel convolutional network provided by the present application;

[0045] Figure 4 A structure diagram of a first branch in a multi-branch pyramid module in a sea image defogging method based on a multi-branch pyramid large kernel convolutional network provided by the application;

[0046] Figure 5 A structure diagram of a second branch in a multi-branch pyramid module in a sea image defogging method based on a multi-branch pyramid large kernel convolutional network provided by the application;

[0047] Figure 6 A sea thick fog scene 3D simulation environment schematic diagram;

[0048] Figure 7 A contrast diagram of effective receptive fields corresponding to different algorithm models;

[0049] Figure 8 An experimental result diagram corresponding to a defogging experiment of different algorithm models;

[0050] Among them, LKEDN-MHP: multi-branch pyramid large kernel encoding-decoding network; PSNR: peak signal-to-noise ratio; SSIM: structural similarity index measure. DETAILED DESCRIPTION

[0051] The preferred embodiments of the application will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0052] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. The meaning of "multiple" is two or more, unless otherwise specifically limited. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0053] Although the embodiments of the application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the application, and the scope of the application is defined by the appended claims and their equivalents.

[0054] Example 1:

[0055] The embodiment of the application provides a sea image defogging method based on a multi-branch pyramid large kernel convolutional network, a flowchart is as shown in Figure 1 The embodiment of the application provides a sea image defogging method based on a multi-branch pyramid large kernel convolutional network, a flowchart is as shown in

[0056] Step 1: obtaining a to-be-processed image;

[0057] Step 2: inputting the to-be-processed image into a pre-trained multi-branch pyramid large kernel encoding-decoding network to obtain a defogged image;

[0058] The multi-branch pyramid large kernel encoding-decoding network comprises a first module, a second module, a multi-branch pyramid module, a self-attention mechanism module and a global path aggregation module.

[0059] Specifically, step 2 comprises:

[0060] The first module is used for depth convolution on the to-be-processed image, and comprises a first-level convolutional layer, a second-level convolutional layer, a third-level convolutional layer and a fourth-level convolutional layer connected in sequence; the number of channels in the first module is 64, and a system architecture diagram is as shown in Figure 2 The input data is an RGB image with a size of 512*512*3, first fog depth estimation sub-information is obtained through a dark channel prior method, and is fused in a channel dimension with an input color image as an input of a local convolutional path; the first module represents a set of layers closest to the input; since the target is a backbone network of a high-performance downstream prediction task model, and the input data size of the downstream prediction task is usually large, it is hoped that more details are captured through several convolutional layers in the initial stage; the number of channels C1 in the first module is 64, after 3*3 and 2*2 down-sampling, a depth 7*7 layer is constructed to capture low-level perception, a 1*1 convolutional layer and another depth 7*7 layer are used for down-sampling, and a depth convolution process diagram is as shown in Figure 3 Different from a conventional convolutional operation, one convolutional kernel of the depth convolution is responsible for one channel, that is, one channel is only convolved by one convolutional kernel; by applying the depth convolution, the calculation amount can be reduced.

[0061] The first convolutional layer is a depth 3*3 convolutional layer;

[0062] The second convolutional layer is a depth 7*7 convolutional layer;

[0063] The third convolutional layer is a depth 1*1 convolutional layer;

[0064] The fourth convolutional layer is a depth 7*7 convolutional layer.

[0065] The second module includes a first processing unit and a second processing unit; an output end of the first processing unit and an input end of the second processing unit are added pixel by pixel; the first processing unit includes batch normalization, a first 1*1 convolution layer, a second 13*13 convolution layer and a third 1*1 convolution layer connected in sequence; the second processing unit includes batch normalization, a fourth 1*1 convolution layer, a Swish activation function and a fifth 1*1 convolution layer connected in sequence;

[0066] The short connection mode and the large depth kernel are used in the second module, and a 1*1 convolution layer is used after the large kernel convolution to increase the depth, because in addition to the ability of the large kernel convolution layer to provide sufficient receptive field and aggregated spatial information, the representation ability of the model is also closely related to the depth, in order to provide more nonlinearity and cross-channel information communication, a 1*1 layer is used to increase the depth in the application, the number of channels C2 in the second module is 128, by using a similar convolution series processing module composed of short connection, batch normalization, two 1*1 convolution layers and Swish activation function, layer normalization is used before the full connection layer, which is beneficial to be fused into the convolution operation for effective inference.

[0067] The multi-branch pyramid module includes a first branch and a second branch;

[0068] The first branch includes a first ladder unit, a first conversion unit, a second ladder unit, a second conversion unit, a third ladder unit, a fourth ladder unit, a first upsampling unit, a fifth ladder unit, a second upsampling unit and a sixth ladder unit connected in sequence;

[0069] The second branch includes a first ladder unit, a first conversion unit, a second ladder unit, a second conversion unit, a third ladder unit, a third conversion unit, a fourth ladder unit, a fifth ladder unit, a first upsampling unit, a sixth ladder unit, a second upsampling unit, a seventh ladder unit, a third upsampling unit and an eighth ladder unit connected in sequence;

[0070] The first branch is used for reflection prediction and shading prediction; the second branch is used for semantic prediction; the reflection and shading prediction tasks provide rich color and texture complementary information for the no-fog image generation task, so that the network can generate high-quality no-fog images with natural color and fine details; wherein the structure diagram of the first branch is as shown in Figure 4 The structure diagram of the second branch is as shown in Figure 5As shown, the first branch includes 6 ladder units, and the encoder and the decoder each include 3 ladder units, wherein each ladder unit includes a plurality of large kernel convolution blocks, each large kernel convolution block uses a 5*5 kernel for reparameterization, and a feedforward convolution network module is placed after each large kernel convolution block, and the second branch includes 8 ladder units and performs pixel-by-pixel fusion on the features of the first branch;

[0071] The self-attention mechanism module includes a first global average pooling layer, a first full connection layer and a second full connection layer; the global average pooling layer, the first full connection layer and the second full connection layer are connected in sequence to generate an attention score of the image to be processed, and based on the attention score, a feature containing attention is obtained; the self-attention mechanism module generates an attention score from the global average pooling layer and two full connection layers, and then the attention score is attached to the output of the fused multi-branch pyramid, so that the model learns to focus on more important information; in order to effectively fuse complementary features, the self-attention mechanism module adaptively enhances appropriate complementary feature channels while weakening irrelevant channels, thereby significantly improving the effectiveness of feature aggregation. The self-attention mechanism module highlights important information while reducing redundant information, further optimizing the performance of feature aggregation.

[0072] The global path aggregation module includes a second global average pooling layer, a third full connection layer, a fourth full connection layer and an up-sampling layer connected in sequence to obtain global features; the global features and the feature containing attention are aggregated pixel by pixel to obtain local-global hybrid features;

[0073] The local-global hybrid features are processed by a global 3*3 convolution layer and global up-sampling to obtain a defogging image;

[0074] The defogging level of the defogging image is determined by combining a loss function, and when the defogging level is greater than a preset threshold, it is determined that the defogging image meets the requirements; the local-global hybrid features are obtained by pixel-by-pixel aggregation of the features obtained by global average pooling, full connection layer and up-sampling and the features containing attention scores. The global path aggregation module is beneficial to improve efficiency while still maintaining the ability to aggregate global information. The mixed output of the local and global paths is converted to the original size by 3*3 convolution and up-sampling to obtain the generated defogging and clarification image. In addition, the discriminator takes the generated image or the real haze-free image as input, obtains the output through the encoder and the full connection, uses the encoder in the multi-branch pyramid to encourage the discriminator to have the same ability to extract and analyze high-level features as the generator, so that the two networks compete with each other to improve their performance;

[0075] The calculation formula corresponding to the combined loss function is as follows:

[0076] L total = alpha1 * L adv + alpha2 * L MSE + alpha3 * L per

[0077] Wherein, the L adv represents the expression of the adversarial loss function; L MSE represents the expression of the MSE loss function; L per represents the expression of the perceptual similarity loss function; alpha1 represents the weight configuration parameter occupied by the adversarial loss; alpha2 represents the weight configuration parameter occupied by the MSE loss; alpha3 represents the weight configuration parameter occupied by the perceptual similarity loss.

[0078] The expression of the adversarial loss function L adv is as follows:

[0079]

[0080] Wherein, B represents the number of samples in a small batch, i=1, 2, …, B; D() represents the output of the discriminator; G() represents the generated image; z represents the input foggy image;

[0081] The expression of the MSE loss function L MSE is as follows:

[0082]

[0083] Wherein, K represents the number of pixels in the generated image, j=1, 2, …, K, and R represents the real fog-free image;

[0084] The expression of the perceptual similarity loss function L per is as follows:

[0085]

[0086] Wherein, P represents the number of elements in the output feature map of Conv3-3 layer in VGG 16 model, n=1, 2, …, P;

[0087] The evaluation loss function of the discriminator is as follows:

[0088]

[0089] By adopting the combination of the adversarial loss, the mean square error loss and the perceptual similarity loss, the application is beneficial to taking into account the pixel quality and human perception, preferably, alpha1=500, alpha2=500, alpha3=500 and alpha4=1.

[0090] ​Before inputting the image to be processed into a pre-trained multi-branch pyramid big kernel encoder-decoder network, the method further includes:

[0091] The image to be processed is processed first-level using the dark channel prior method to obtain fog depth estimation sub-information of the image to be processed;

[0092] The fog depth estimation information of the image to be processed and the image to be processed are fused to obtain a first-level input image;

[0093] The first-level input image is input into a pre-trained multi-branch pyramid big-kernel encoder-decoder network;

[0094] In one specific embodiment, a 3D simulation environment for a dense fog scene at sea was constructed, such as... Figure 6 As shown, in each pair of images, the left side represents the target sample under dense fog conditions, and the right side represents the target sample under clear, fog-free conditions. A total of 18,540 images were collected at different angles, with different target sizes and under different lighting conditions. Each image is 1920×1080 pixels in size, including 9,648 images under dense fog conditions and 8,892 clear images. To improve the learning ability of the network, a data augmentation method was adopted: First, each training image was randomly cropped to produce 5 image blocks with the same aspect ratio as the original image; then, a horizontal flipping method was used to double the number of training samples, while creating new geometric texture variations for the training images. To verify the universality of the proposed LKEDN-MHP, an additional public dataset of outdoor dense fog scenes, O-Haze, was added.

[0095] Model training and testing

[0096] Training and testing were performed on a GeForce GTX TITAN X graphics card. The network input size was 512×512, and the training iterations were 1000. The learning rate was fixed at 10% for the first 800 weeks. -4 In the last 200 cycles, we linearly decayed the learning rate to zero. We used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to measure the dehazing performance of LKEDN-MHP. Setting the batch size to 1 resulted in better results in image restoration tasks. In the multi-branch pyramid, each step module contains three architectural hyperparameters: the number of large kernel convolutional blocks, channel dimension, and kernel size. The model containing these parameters is called LKEDN-MHP-B (B for Base). Based on this, a wider model was designed called LKEDN-MHP-L (L for Large). The architectural hyperparameter settings for each step module in the experiments are shown in Table 1.

[0097] Table 1 Architecture Hyperparameter Settings

[0098]

[0099] Through the algorithm of the present application, the performance on the self-built simulation data set and the O-Haze data set is compared with other currently most advanced algorithms, in order to better reflect the superiority of the algorithm proposed in this paper, the most advanced algorithm based on CNN (EDN-GTM) and the most advanced algorithm based on ViT (DehazeFormer) are selected as representatives for experimental comparison, the effective receptive field comparison diagram corresponding to different algorithm models is as shown in Figure 7 The larger the area of the bright color region, the larger the effective receptive field, and the absolute value of the gradient relative to the input is measured at the center position of the feature map, for 32 randomly selected foggy input images, the average value is taken on all channels of each feature map, and Figure 7 It can be known that the effective receptive field constructed by the large kernel convolution is the largest.

[0100] The dehazing experiment is carried out on each algorithm model, and the corresponding experimental result diagram is as shown in Figure 8 The quantitative dehazing result is as shown in Table 2,

[0101] Table 2 Quantitative dehazing results of self-built data set and O-HAZE data set

[0102]

[0103] The dehazing effect of the present application in the self-built data set and the O-Haze data set is optimal, as shown in Table 2, in the quantitative experimental result of the self-built offshore thick fog image data set obtained in the simulation environment based on digital twinning, the PSNR of the LKEDN-MHP-B proposed in the present application reaches 25.79dB, and the best offshore image dehazing performance is realized, on this basis, the PSNR of the LKEDN-MHP-L of the present scheme is further improved by 2.39dB, and the SSIM index also reaches the optimal value of 0.8856, therefore, the LKEDN-MHP algorithm proposed in the present application has achieved good results on the self-built simulation data set and the real outdoor thick fog scene data set. The results show that the LKEDN-MHP scheme proposed has a well-designed architecture and can effectively perform the offshore image haze removal task.

[0104] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program code.

[0105] The present application is described in reference to the drawings using a flowchart and / or a block diagram of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing machine, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0106] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0108] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for dehazing marine images based on a multi-branch pyramidal large-kernel convolutional network, characterized in that, include: Obtain the image to be processed; The image to be processed is input into a pre-trained multi-branch pyramid big kernel encoder-decoder network to obtain a dehazed image; The big-core encoding and decoding network of the branch pyramid includes a first module, a second module, a multi-branch pyramid module, a self-attention mechanism module, and a global path aggregation module. The first module is used to perform depthwise convolution on the image to be processed, including a first-level convolutional layer, a second-level convolutional layer, a third-level convolutional layer, and a fourth-level convolutional layer connected in sequence; the number of channels in the first module is 64; The second module includes a first processing unit and a second processing unit; the output of the first processing unit and the input of the second processing unit are added pixel by pixel; the first processing unit includes a batch normalization, a first 1×1 convolutional layer, a second 13×13 convolutional layer and a third 1×1 convolutional layer connected in sequence; the second processing unit includes a batch normalization, a fourth 1×1 convolutional layer, a Swish activation function and a fifth 1×1 convolutional layer connected in sequence; The multi-branch pyramid module includes a first branch and a second branch; The first branch includes a first step unit, a first conversion unit, a second step unit, a second conversion unit, a third step unit, a fourth step unit, a first upsampling unit, a fifth step unit, a second upsampling unit, and a sixth step unit connected in sequence; The second branch includes a first-level step unit, a first-level conversion unit, a second-level step unit, a second-level conversion unit, a third-level step unit, a third-level conversion unit, a fourth-level step unit, a fifth-level step unit, a first-level upsampling unit, a sixth-level step unit, a second-level upsampling unit, a seventh-level step unit, a third-level upsampling unit, and an eighth-level step unit connected in sequence. The self-attention mechanism module includes a first global average pooling layer, a first fully connected layer, and a second fully connected layer; the global average pooling layer, the first fully connected layer, and the second fully connected layer are connected in sequence to generate an attention score for the image to be processed, and based on the attention score, features containing attention are obtained; The global path aggregation module includes a second global average pooling layer, a third fully connected layer, a fourth fully connected layer, and an upsampling layer connected in sequence to obtain global features; the global features and the attention-included features are aggregated pixel by pixel to obtain local-global hybrid features; The local-global hybrid features are processed through a global 3x3 convolutional layer and global upsampling to obtain a dehazed image; The dehazing level of the dehazed image is determined by combining loss functions. When the dehazing level is greater than a preset threshold, the dehazed image is determined to meet the requirements.

2. The method as described in claim 1, characterized in that, The first convolutional layer is a 3×3 convolutional layer; The second convolutional layer is a 7×7 convolutional layer; The third convolutional layer is a 1×1 convolutional layer; The fourth convolutional layer is a 7×7 convolutional layer.

3. The method as described in claim 1, characterized in that, The formula for calculating the combined loss function is as follows: ； Among them, the This represents the expression for the adversarial loss function; Represent the MSE loss function expression; This represents the expression for the perceptual similarity loss function; This indicates the weighting parameter for adversarial losses; The weighting parameter represents the proportion of MSE loss. This represents the weighting parameter for the perceived similarity loss.

4. The method as described in claim 1, characterized in that, The adversarial loss function expression as follows: ； in, This indicates the number of samples in a mini-batch. ; This represents the output of the discriminator; This represents the generated image; This represents an input image with fog. The MSE loss function expression as follows: ； in, This indicates the number of pixels in the generated image. , Represents a true, fog-free image; The expression for the perceptual similarity loss function as follows: ； in, This represents the output feature map of Conv3-3 layer in the VGG 16 model. The number of elements in .

5. The method as described in claim 1, characterized in that, Before inputting the image to be processed into a pre-trained multi-branch pyramid big kernel encoder-decoder network, the method further includes: The image to be processed is processed first-level using the dark channel prior method to obtain fog depth estimation sub-information of the image to be processed; The fog depth estimation information of the image to be processed and the image to be processed are fused to obtain a first-level input image; The first-level input image is input into a pre-trained multi-branch pyramid big kernel encoder-decoder network.

Images