An image defogging method based on a self-supervised feature fusion GAN model

Abstract

The application relates to an image defogging method based on a self-supervised feature fusion GAN model, the self-supervised feature fusion GAN model comprises a discriminator and a generator based on a U-net architecture, the defogging network comprises a trained generator, the generator comprises connected encoders and decoders; the encoders perform downsampling operations, and the decoders perform upsampling operations. The encoder comprises sequentially connected convolution layers and multistage first processing layers, the first processing layer comprises sequentially connected residual layers, convolution layers and feature fusion modules; the decoder comprises sequentially connected multistage second processing layers, a self-supervised attention module and convolution layers, and the second processing layer comprises sequentially connected deconvolution layers, an enhancement strategy module and a residual layer. The defogging method based on the above structure improves the accuracy and detail retention capability of generated images, and the generated defogging images are more real and clear.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image defogging, and particularly relates to an image defogging method based on a self-supervised feature fusion GAN model. BACKGROUND

[0002] When an airplane flies in the high altitude, the airplane surface will be covered with ice due to the contact with water vapor in the air, and the airplane icing will greatly affect the flight safety. The icing wind tunnel can provide low temperature, low pressure, high wind speed and cloud field, so as to simulate the real icing environment when the airplane passes through the cloud layer, and is an important means for measuring the influence of icing on the flight performance of the airplane and verifying the effectiveness of the icing protection measures. However, during the test process, there are suspended water droplets with certain liquid water content in the test section, and when the light in the test section passes through the water droplets, the generated image is blurred due to the scattering and absorption of the light, which leads to the decline of the picture quality. These pictures usually have color distortion, contrast reduction, loss of edge and texture information, and the situation of the icing place cannot be accurately observed, so it is necessary to perform the defogging processing on the image.

[0003] In order to solve the technical problem of the foggy image, the related scholars proposed a data-based deep learning defogging method. However, simply stacking the network layers cannot bring satisfactory defogging effect, and it is necessary to design a better image defogging network model and solve the problem of unsatisfactory defogging effect of the prior art based on the network model. SUMMARY

[0004] In order to solve the technical problem of the unsatisfactory defogging effect of the data-based deep learning defogging model in the prior art, the present application provides an image defogging method based on a self-supervised feature fusion GAN model. Specifically as follows:

[0005] In the first aspect, an image defogging method based on a self-supervised feature fusion GAN model is provided in an embodiment;

[0006] The image defogging method is realized based on an image defogging network, and the image defogging network is obtained by training a self-supervised feature fusion GAN model;

[0007] The self-supervised feature fusion GAN model comprises a discriminator and a generator based on a U-net architecture; the defogging network comprises the trained generator, the generator comprises a connected encoder and decoder; the encoder performs downsampling operation, and the decoder performs upsampling operation;

[0008] The encoder comprises a convolution layer and a first processing layer connected in sequence, the first processing layer comprises a residual layer, a convolution layer and a feature fusion module connected in sequence; the decoder comprises a second processing layer, a self-supervised attention module and a convolution layer connected in sequence, the second processing layer comprises an inverse convolution layer, an enhancement strategy module and a residual layer connected in sequence;

[0009] The image defogging method comprises: data preprocessing on the foggy image, resetting the image to a preset pixel size to obtain a to-be-processed object; inputting the to-be-processed object into the generator to sequentially undergo processing of the encoder and processing of the decoder to obtain a defogged image;

[0010] The processing of the encoder comprises:

[0011] The to-be-processed object is obtained.

[0012] The to-be-processed object is subjected to convolution downsampling processing to obtain a first feature map, and the first feature map is subjected to processing of a first processing layer; the processing of the first processing layer comprises:

[0013] For the first processing layer, the processing method comprises: after residual processing of the first feature map, a first residual feature map of the first processing layer is obtained; after convolution downsampling processing of the first residual feature map, a first downsampling feature map of the first processing layer is obtained; after feature fusion processing of the first feature map and the first downsampling feature map, a first fusion feature map of the first processing layer is obtained.

[0014] For other first processing layers except the first processing layer, the processing method comprises: after residual processing of the fusion feature map obtained by the previous first processing layer, a residual feature map of the first processing layer is obtained; after convolution downsampling processing of the residual feature map, a downsampling feature map of the first processing layer is obtained; after feature fusion processing of the downsampling feature map of the first processing layer, the first feature map and the fusion feature maps obtained by all previous first processing layers, a fusion feature map of the first processing layer is obtained.

[0015] The processing of the decoder comprises:

[0016] The fusion feature map output by the encoder is obtained, and for the second processing layer, the processing of any second processing layer comprises:

[0017] The input feature map is subjected to deconvolution upsampling processing to obtain an upsampled feature map of the level; the upsampled feature map of the level and a residual feature map of a corresponding level in the first processing layer are subjected to enhanced feature fusion processing to obtain an enhanced feature map of the level, the corresponding level being a central-symmetry corresponding level, i.e., the second processing layer of the first level in the decoder corresponds to the first processing layer of the last level in the encoder, and the second processing layer of the last level in the decoder corresponds to the first processing layer of the first level in the encoder; the enhanced feature map of the level is subjected to residual processing to obtain a residual feature map of the level.

[0018] The residual feature map of the last level is obtained, subjected to self-supervised attention processing to obtain a second feature map; the second feature map is subjected to convolution processing to obtain the defogged image.

[0019] In an embodiment, the discriminator is a Patch-GAN discriminator.

[0020] In an embodiment, the residual layers in the first processing layer and / or the second processing layer include two groups of sequentially connected residual processing layers, each group of residual processing layers including sequentially connected Conv3x3 layers, BN layers and Relu layers.

[0021] In an embodiment, the feature fusion processing in the first processing layer of the nth level includes:

[0022]

[0023] wherein m n is the downsampled feature map of the nth level, p is the first feature map, is a fusion feature map obtained from the first processing layers of all previous levels, denotes a feature fusion module of the nth level, 1≤n≤N, and N is the total number of levels of the first processing layer.

[0024] In an embodiment, the enhanced feature fusion processing in the second processing layer of the nth level includes:

[0025]

[0026] wherein i n is the residual feature map of the first processing layer of the nth level, (j n )↑2 is the upsampled feature map of the second processing layer of the nth level, denotes a convolution unit, i n +(j n )↑2 denotes an enhanced feature.

[0027] In an embodiment, the image defogging network is obtained based on training of a self-supervised feature fusion GAN model, including: first, based on a mean loss function L Res, a perceptual loss function L P and a binary cross-entropy function L A The self-supervised feature fusion GAN model is trained, and then a total loss function L all The self-supervised feature fusion GAN model is standardized, L all =L Res +α1L P +α2L A , wherein α1 represents the weight of the perceptual loss function L P , and α2 represents the weight of the binary cross-entropy function L A .

[0028] In an embodiment, the mean loss function L Res includes:

[0029]

[0030] wherein I i represents an input foggy image, J i represents a real fog-free image corresponding to the image, G(I i ) represents a de-fogging image generated by the generator, and M represents the number of training data.

[0031] In an embodiment, the perceptual loss function L P includes:

[0032]

[0033] wherein φ(·) represents a feature map obtained from a VGG16 network layer.

[0034] In an embodiment, the binary cross-entropy function L A includes:

[0035]

[0036] wherein D(·) represents a discriminator in the network.

[0037] The present application has the following advantages:

[0038] Through the structure of the encoder, the input non-adjacent layer features are used to achieve the effect of feature fusion and enhancement, the low-level features, high-level features and middle-level features are combined, that is, the edge features and texture features are integrated, so as to better capture the details and structure of the image at this stage, through the structure of the decoder, the spatial dimension of the input feature map is increased, so as to restore the lost detail information, gradually restore the size of the feature map to the size of the original image, and learn a specific nonlinear transformation to optimize the decoder output, further correct and refine the output, improve the accuracy and detail retention capability of the generated image, and generate a more real and clear de-fogging image. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a schematic diagram of the overall structure of a self-supervised feature fusion GAN model according to an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of the structure of a generator according to an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of the structure of a residual layer according to an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of the structure of an enhancement strategy module according to an embodiment of the present application;

[0043] Figure 5 is a schematic diagram of the structure of a self-supervised attention module according to an embodiment of the present application;

[0044] Figure 6 is a fog removal effect diagram of a wind tunnel test picture according to an embodiment of the present application;

[0045] Figure 7 is a comparison diagram of the fog removal effects of the fog removal method according to an embodiment of the present application and other methods on the HSTS test set. DETAILED DESCRIPTION

[0046] The specific embodiments of the present application will be further described below in conjunction with the accompanying drawings, so as to have a further understanding of the concept of the present application, the technical problems solved, the technical features constituting the technical solutions and the technical effects brought by the present application. However, it should be noted that the description of these embodiments is illustrative and does not constitute a specific limitation of the present application.

[0047] The image fog removal method of the present application is implemented based on an image fog removal network, which is obtained by training a self-supervised feature fusion GAN model. Please refer to Figure 1 , the self-supervised feature fusion GAN model includes a discriminator and a generator based on a U-net architecture; the fog removal network includes the trained generator, and the generator includes a connected encoder and a decoder; the encoder performs downsampling operation, and the decoder performs upsampling operation.

[0048] In the training phase of the image fog removal network, the generator continuously generates pictures close to the haze-free pictures from the foggy pictures, the discriminator discriminates whether the picture is true or false, and then returns the result to the generator. Through continuous training, more effective features are extracted to reduce errors, and the iterative process is equivalent to a zero-sum game process, thereby continuously optimizing the generator, so that the generated fog removal pictures by the generator are increasingly close to the real haze-free pictures. After training is completed, the image fog removal network implementing the fog removal method of the present application is obtained.

[0049] In an embodiment, the discriminator is a Patch-GAN discriminator. Compared with other discriminators, the output of the Patch-GAN is a matrix, and each element in the matrix represents a relatively large receptive field in the original image, corresponding to a patch in the original image. Therefore, more attention can be paid to the details of the image during training of the model, and a higher resolution image can be obtained.

[0050] In an embodiment, during training, first, the self-supervised feature fusion GAN model is trained based on a mean loss function L Res , a perception loss function L P , and a binary cross-entropy function L A respectively, and then the self-supervised feature fusion GAN model is standardized by using a total loss function L all , L all =L Res +α1L P +α2L A , where α1 represents the weight of the perception loss function L P , and α2 represents the weight of the binary cross-entropy function L A .

[0051] In an embodiment, the mean loss function L Res includes:

[0052]

[0053] where I i represents an input foggy image, J i represents a real fog-free image corresponding to the image, G(I i ) represents a de-fogging image generated by the generator, and M represents the number of training data.

[0054] In an embodiment, the perception loss function L P includes:

[0055]

[0056] where φ(·) represents a feature map obtained from a VGG16 network layer.

[0057] In an embodiment, the binary cross-entropy function L A includes:

[0058]

[0059] where D(·) represents a discriminator in the network.

[0060] Based on the above, in an embodiment of the present application, first, the mean loss function L Resto compare the difference between the generated defogging picture and the real fog-free picture in the image pixel space; the perceptual similarity of the defogging picture and the fog-free picture in the feature space is evaluated by using the parameters of the VGG16 pre-training network (a classic network) model; in order to restore the authenticity of the image, a binary classification cross-entropy function is used to calculate the loss value. Finally, the reconstruction loss function, the perceptual loss function and the adversarial loss function are combined to form a total loss function L all The specification self-supervised feature fusion GAN model.

[0061] Please refer to Figure 2 The encoder includes a convolutional layer and a plurality of first processing layers connected in sequence, and the first processing layer includes a residual layer, a convolutional layer and a feature fusion module connected in sequence; the decoder includes a plurality of second processing layers, a self-supervised attention module and a convolutional layer connected in sequence, and the second processing layer includes a deconvolutional layer, an enhancement strategy module and a residual layer connected in sequence.

[0062] In the defogging method, first, the foggy image is preprocessed, and the image is reset to a preset pixel size to obtain a to-be-processed object. In an embodiment of the present application, the image is reset to 1024x1024 pixel size. Then the to-be-processed object is input into the generator for processing by the encoder and the decoder in sequence to obtain a defogging image.

[0063] In the encoder, the main purpose is to reduce the spatial dimension of the feature map while extracting important abstract features of the foggy image. The existing technology can be used to achieve this, and the present application provides a new processing method. The processing of the encoder includes: obtaining the to-be-processed object; performing convolutional downsampling processing on the to-be-processed object to obtain a first feature map, and performing multi-level first processing layer processing on the first feature map.

[0064] For the processing of the multi-level first processing layer, it includes:

[0065] For the first processing layer, the processing method includes: after the first feature map is processed by residual processing, a first residual feature map is obtained; after the first residual feature map is processed by convolutional downsampling, a first downsampling feature map is obtained; after the first feature map and the first downsampling feature map are processed by feature fusion, a first fusion feature map is obtained.

[0066] For the first processing layer other than the first level, the processing method includes: after the fusion feature map obtained by the previous first processing layer is processed by residual processing, a residual feature map of this level is obtained; after the residual feature map of this level is processed by convolutional downsampling, a downsampling feature map of this level is obtained; after the downsampling feature map of this level, the first feature map and the fusion feature maps obtained by all previous first processing layers are processed by feature fusion, a fusion feature map of this level is obtained.

[0067] In an embodiment of the present application, the feature fusion processing in the first processing layer of the nth stage includes:

[0068]

[0069] wherein m n is the down-sampled feature map of the nth stage, p is the first feature map, is the fusion feature map obtained from all previous first processing layers, taking the feature fusion processing in the third first processing layer as an example, the fusion feature maps obtained from all previous first processing layers include and denotes the feature fusion module of the nth stage, 1≤n≤N, and N is the total number of stages of the first processing layer.

[0070] In the above processing scheme of the encoder, the foggy image to be processed is first input to the first convolutional layer Conv for down-sampling operation to capture low-level features in the image, including preliminary edge features and other low-level features (such as color features). The feature map obtained after convolution is input to the residual block Res in the first first processing layer, which learns low-level feature representation by increasing the residual block, further processes and enhances the input feature map, and stabilizes the gradient vanishing problem caused by the increase of network layers.

[0071] Each residual block is usually composed of multiple convolutional layers, including 1x1 and 3x3 convolution operations, activation functions and batch normalization. For the residual layer in the first processing layer, please refer to Figure 3 , which includes two groups of sequentially connected residual processing layers, and each group of residual processing layers includes Conv3x3 layers, BN layers and Relu layers connected in sequence. Through the residual connection, the input features are added to the output features, so that the model can learn the subtle details in the image. The output feature map of the residual layer is then input to the convolutional layer Conv for down-sampling operation to reduce the spatial dimension of the feature map and extract higher-level texture features of the object to be processed. The output feature map is then transmitted to the feature fusion module; the input of the feature fusion module DFF includes the output of the previous layer, the first convolutional layer Conv and the output of each feature fusion module DFF in the front, and through the DFF module, the input of non-adjacent layers can be used to achieve the effect of feature fusion and enhancement, combining low-level features, high-level features and intermediate-level features, i.e. edge features and texture features, to better capture the details (texture features) and structures (edge features) of the image at this stage. In the embodiment shown in Figure 2 , the encoder continuously optimizes the extracted features by using four first processing layers.

[0072] The feature fusion module is located after the convolutional layer, which can better extract the high-frequency information of the foggy image through the feedback mechanism, slowly fuse the extracted feature difference into the down-sampled latent feature, repair the missing spatial information, and further enhance the enhanced features at the current level through the error feedback mechanism.

[0073] The processing of the decoder includes:

[0074] The fusion feature map output by the encoder is obtained. For the processing of any level of the second processing layer, the processing includes: performing up-sampling processing on the input feature map through de-convolution to obtain the up-sampled feature map of the level; performing enhanced feature fusion processing on the up-sampled feature map of the level and the residual feature map of the corresponding level in the first processing layer to obtain the enhanced feature map of the level, the corresponding level being a center-symmetric corresponding level, that is, the first level of the second processing layer in the decoder corresponds to the last level of the first processing layer in the encoder, and the last level of the second processing layer in the decoder corresponds to the first level of the first processing layer in the encoder; performing residual processing on the enhanced feature map of the level to obtain the residual feature map of the level.

[0075] In an embodiment, please refer to Figure 4 The enhanced feature fusion processing in the n-th level of the second processing layer includes:

[0076]

[0077] Wherein, i n is the residual feature map of the n-th level of the first processing layer, (j n )↑2 is the up-sampled feature map of the n-th level of the second processing layer, and ↑2 represents up-sampling with a scale of 2. represents a convolution unit, i n +(j n )↑2 represents enhanced features.

[0078] Taking an example in which both the first processing layer and the second processing layer have four levels, if the levels of the first processing layer are the first level, the second level, the third level and the fourth level in order from front to back according to the processing direction, then the levels of the second processing layer are the fourth level, the third level, the second level and the first level in order from front to back according to the processing direction.

[0079] In the decoder, the spatial dimension of the input feature map is increased by the upsampling layer to recover the lost detail information, gradually restoring the size of the feature map to the size of the original image. The deconvolution layer Deconv performs the upsampling operation, and the input of the first deconvolution layer Deconv in the network structure is the output of the last layer of the encoder, and the output feature map is transmitted to the enhancement strategy module S0S; the input of the enhancement strategy module S0S also has the latent feature map (the output of the residual layer) corresponding to the layer in the encoder, so as to achieve the effect of enhancing the features of the haze-free picture.

[0080] The enhanced feature map output by the enhancement strategy module S0S is input into the residual layer. In an embodiment of the present application, for the residual layer in the first processing layer, please refer to Figure 3 , which includes two groups of residual processing layers connected in sequence, and each group of residual processing layers includes Conv3x3 layers, BN layers and Relu layers connected in sequence. In this embodiment, since the residual block contains multiple convolution layers and activation functions, it can optimize the output of the decoder by learning a specific nonlinear transformation, further correct and refine the output, improve the accuracy and detail retention capability of the generated image, and generate a more real and clear defogging image.

[0081] The last level of residual feature map is obtained, and the second feature map is obtained after self-supervised attention processing; the second feature map is convoluted to obtain the defogging image.

[0082] Please refer to Figure 5 , the self-supervised attention module can provide GT signals useful for progressive image restoration, and can suppress features with less current information by generating attention maps, extract useful feature quantities in the hazy image and pass them down, and continuously refine the features.

[0083] In this way, through the structure of the encoder, the input non-adjacent layer features are used to achieve the effects of feature fusion and enhancement, the low-level features, high-level features and middle-level features are combined, that is, the edge features and texture features are integrated, so as to better capture the details and structures of the image at this stage, through the structure of the decoder, the spatial dimension of the input feature map is increased to recover the lost detail information, gradually restoring the size of the feature map to the size of the original image, and learning a specific nonlinear transformation to optimize the output of the decoder, further correcting and refining the output, improving the accuracy and detail retention capability of the generated image, and generating a more real and clear defogging image.

[0084] To verify the effectiveness of the image defogging method based on the self-supervised feature fusion GAN model provided in the application, the RESIDE dataset is used in the application, which is a relatively large public dataset. The training set includes 110500 pairs of synthetic indoor foggy images (ITS) and 313950 pairs of synthetic outdoor foggy images (OTS). The comprehensive target test set (SOTS) contains 500 indoor images and 500 outdoor images, and the mixed subjective test set (HSTS) contains 10 real foggy images and 10 synthetic images. The wind tunnel test picture and the real outdoor foggy picture in the HSTS are tested, and the effect diagram is as shown in Figure 6 It can be seen from Figure 6 that the application achieves very good results in the thin fog scene. Even in the thick fog case which greatly hinders the image restoration of many pictures, the defogging effect of the application is also relatively good.

[0085] The application also compares the performance of the existing common model and the application in the HSTS real outdoor foggy picture. The existing model selected is DCPDN, AOD-Net and MSCNN. The defogging effects of the application and the existing model are as shown in Figure 7 It can be seen from Figure 7 that the defogging effect of the application is better, and more details of the foggy picture can be restored, such as the truck on the bridge in the first test picture. The method of the application can better restore the clear outline and color of the truck.

[0086] In summary, the image defogging method based on the self-supervised feature fusion GAN model provided in the application has good defogging effect, and its effect is better than other defogging models commonly used in the prior art.

[0087] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can modify the technical solutions described in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. An image defogging method based on a self-supervised feature fusion GAN model, characterized in that: the image defogging method is realized based on an image defogging network, and the image defogging network is obtained by training a self-supervised feature fusion GAN model; the self-supervised feature fusion GAN model comprises a discriminator and a generator based on a U-net architecture; the defogging network comprises the trained generator, and the generator comprises a connected encoder and a decoder; the encoder performs downsampling operation, and the decoder performs upsampling operation; the encoder comprises a convolutional layer and a plurality of first processing layers connected in sequence, and the first processing layer comprises a residual layer, a convolutional layer and a feature fusion module connected in sequence; the decoder comprises a plurality of second processing layers, a self-supervised attention module and a convolutional layer connected in sequence, and the second processing layer comprises a deconvolutional layer, an enhancement strategy module and a residual layer connected in sequence; the image defogging method comprises: performing data preprocessing on a foggy image, resetting the image to a preset pixel size to obtain a to-be-processed object, inputting the to-be-processed object into the generator, and sequentially performing processing of the encoder and processing of the decoder to obtain a defogged image; the processing of the encoder comprises: obtaining the to-be-processed object; performing convolutional downsampling processing on the to-be-processed object to obtain a first feature map, and performing processing of the first processing layer on the first feature map; for the processing of the first processing layer, the processing method comprises: for the first processing layer, the processing method comprises: performing residual processing on the first feature map to obtain a residual feature map of the first processing layer; performing convolutional downsampling processing on the residual feature map of the first processing layer to obtain a downsampling feature map of the first processing layer; and performing feature fusion processing on the first feature map and the downsampling feature map of the first processing layer to obtain a fusion feature map of the first processing layer; for other first processing layers except the first processing layer, the processing method comprises: performing residual processing on the fusion feature map obtained by the previous first processing layer to obtain a residual feature map of the first processing layer; performing convolutional downsampling processing on the residual feature map of the first processing layer to obtain a downsampling feature map of the first processing layer; and performing feature fusion processing on the downsampling feature map of the first processing layer, the first feature map and the fusion feature maps obtained by all previous first processing layers to obtain a fusion feature map of the first processing layer; the processing of the decoder comprises: obtaining the fusion feature map output by the encoder, and for the processing of the second processing layer, the processing of any second processing layer comprises: performing deconvolutional upsampling processing on the input feature map to obtain an upsampling feature map of the second processing layer; performing enhanced feature fusion processing on the upsampling feature map of the second processing layer and the residual feature map of the corresponding first processing layer to obtain an enhanced feature map of the second processing layer, wherein the corresponding first processing layer is a center-symmetric corresponding first processing layer, that is, the first processing layer of the last level in the encoder corresponds to the second processing layer of the first level in the decoder, and the first processing layer of the first level in the encoder corresponds to the second processing layer of the last level in the decoder; and performing residual processing on the enhanced feature map of the second processing layer to obtain a residual feature map of the second processing layer. The last-stage residual feature map is obtained, and a second feature map is obtained after self-supervised attention processing; and the second feature map is subjected to convolution processing to obtain the defogging image.

2. The image defogging method based on the self-supervised feature fusion GAN model according to claim 1, wherein: The discriminator is a Patch-GAN discriminator.

3. The image defogging method based on the self-supervised feature fusion GAN model according to claim 1, wherein: The residual layer in the first processing layer and / or the second processing layer includes two groups of residual processing layers connected in sequence, and each group of residual processing layers includes a Conv3x3 layer, a BN layer and a Relu layer connected in sequence.

4. The image defogging method based on the self-supervised feature fusion GAN model according to claim 1, wherein: The feature fusion processing in the first processing layer of the nth stage includes: wherein m n is the down-sampled feature map of the n-th stage, p is the first feature map, is the fusion feature map obtained from the first processing layer of all previous stages, denotes the feature fusion module of the n-th stage, 1≤n≤N, and N is the total number of stages of the first processing layer.

5. The image defogging method based on the self-supervised feature fusion GAN model according to claim 4, wherein: The enhanced feature fusion processing in the second processing layer of the nth stage includes: wherein i n is the residual feature map of the n-th first processing layer, (j n )↑2 is the up-sampling feature map of the n-th second processing layer, denotes a convolution unit, i n +(j n )↑2 denotes an enhanced feature.

6. The image defogging method based on the self-supervised feature fusion GAN model according to claim 1, wherein: The image defogging network is obtained by training a self-supervised feature fusion GAN model, including: firstly, training the self-supervised feature fusion GAN model based on a mean loss function L Res , a perception loss function L P and a binary cross-entropy function L A respectively, then adopting a total loss function L all to standardize the self-supervised feature fusion GAN model, L all =L Res +α1L P +α2L A , wherein α1 represents the weight of the perception loss function L P , and α2 represents the weight of the binary cross-entropy function L A .

7. The image defogging method based on the self-supervised feature fusion GAN model according to claim 6, wherein: Mean loss function L Res Comprising: where I i represents the input foggy image, J i represents the real fog-free image corresponding to the image, G(I i ) represents the dehazed image generated by the generator, and M represents the number of training data.

8. The image defogging method based on the self-supervised feature fusion GAN model according to claim 7, wherein, Perceptual loss function L P comprises: wherein, φ(·) represents a feature map obtained from a VGG16 network layer.

9. The image defogging method based on the self-supervised feature fusion GAN model according to claim 7, wherein, Binary cross-entropy function L A comprises: wherein, D(·) represents a discriminator in the network.

Images