Dehazing method based on bidirectional GAN ​​network and bidirectional GAN ​​network model

Abstract

The present application discloses a defogging method based on a bidirectional GAN ​​network and a bidirectional GAN ​​network model. The method includes using the extracted depth information corresponding to different pixels in the image to be defogged to perform depth processing on the image to be defogged, and then inputting it into a pre-built bidirectional GAN ​​network model to obtain a defogged image output by the model. In the model, the clear image input port, the first generation unit and the first discriminator are connected in sequence for feature extraction and reconstruction of the clear image, and the fog image input port, the second generation unit and the second discriminator are connected in sequence for feature extraction and reconstruction of the fog image; the shared latent space stores high-level features extracted for the clear image and high-level features extracted for the fog image. The bidirectional GAN ​​network model is trained and verified through paired clear images and fog images, and includes a bidirectional mapping relationship between the fog domain and the clear domain. It can process images in different domains and effectively ensure the authenticity of the image reconstruction.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defogging method based on a bidirectional GAN ​​network and a bidirectional GAN ​​network model. Background Art

[0002] Fog and haze reduce the clarity of images captured by cameras, making it difficult for computers to recognize the features of objects in the images. In order to obtain truly clear images, dehazing the images is essential.

[0003] The most common method for image defogging is based on machine learning. It mainly uses convolutional neural networks to learn the mapping relationship between foggy and clear images through a large amount of training data, and then defogging the foggy images. The commonly used defogging convolutional neural network is the Generative Adversarial Network (GAN). This network model uses an unsupervised learning method and produces high-accuracy output through the mutual game learning of two modules: the generator module and the discriminator module. The generator module includes an encoder and a decoder, such as Figure 1 As shown in Figure 1, the encoder is used to extract the feature vector from the input image, and the decoder is used to restore the low-level features from the feature vector.

[0004] However, currently, GAN is used for the one-way conversion from foggy images to clear images. Therefore, GAN only contains a one-way mapping relationship from the foggy domain to the clear domain. After the foggy image is processed by the encoder and decoder, the output clear image usually has halo effects and artificial defects, and the original image information is poorly preserved. Summary of the Invention

[0005] In order to solve the problem that the current GAN only contains a one-way mapping relationship from the foggy domain to the clear domain, resulting in halo effects and artificial defects in the output of the final image, and poor preservation of the original image information, the present application discloses a defogging method based on a bidirectional GAN ​​network and a bidirectional GAN ​​network model through the following embodiments.

[0006] In a first aspect, the present application discloses a defogging method based on a bidirectional GAN ​​network, the defogging method comprising:

[0007] Extract the depth information corresponding to different pixels in the image to be defogged;

[0008] Performing depth processing on the image to be defogged according to the depth information;

[0009] The image to be defogged after deep processing is input into a pre-built bidirectional GAN ​​network model, wherein the bidirectional GAN ​​network model includes an input module, a generation module and a discriminant module; the input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discriminant module includes a first discriminator and a second discriminator; the clear image input port, the first generation unit and the first discriminator are connected in sequence for extracting and reconstructing features for the clear image, and the fog image input port, the second generation unit and the second discriminator are connected in sequence for extracting and reconstructing features for the fog image; the first generation unit includes a first encoder, a shared latent space and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space and a second decoder connected in sequence; the shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder, the high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the fog image; the training set and the validation set of the bidirectional GAN ​​network model both include paired clear images and fog images;

[0010] Obtain the dehazed image output by the bidirectional GAN ​​network model.

[0011] Optionally, extracting depth information corresponding to different pixels in the image to be defogged includes:

[0012] Performing format conversion on the image to be defogged, and extracting the brightness and saturation of the image to be defogged;

[0013] Depth information corresponding to different pixels in the image to be defogged is generated according to the brightness and saturation.

[0014] The second aspect of the present application discloses a bidirectional GAN ​​network model, which includes an input module, a generation module and a discrimination module;

[0015] The input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discrimination module includes a first discriminator and a second discriminator;

[0016] The clear image input port, the first generation unit, and the first discriminator are connected in sequence for extracting and reconstructing features for the clear image; the fog image input port, the second generation unit, and the second discriminator are connected in sequence for extracting and reconstructing features for the fog image;

[0017] The first generation unit includes a first encoder, a shared latent space, and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space, and a second decoder connected in sequence;

[0018] The shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder, wherein the high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the foggy image;

[0019] The training set and validation set of the bidirectional GAN ​​network model both include paired clear images and foggy images.

[0020] Optionally, the first encoder and the second encoder respectively include a first convolution block, a second convolution block and a first coupled residual block connected in sequence;

[0021] The first decoder and the second decoder respectively include a second coupled residual block, a first stride convolution block and a second stride convolution block connected in sequence;

[0022] An output terminal of the first coupled residual block is connected to an input terminal of the shared latent space, and an output terminal of the shared latent space is connected to an input terminal of the second coupled residual block;

[0023] The output end of the first convolution block is jump-connected to the input end of the second stride convolution block;

[0024] The output end of the second convolution block is jump-connected to the input end of the first-step convolution block.

[0025] Optionally, the first coupled residual block and the second coupled residual block are respectively composed of a plurality of cascaded sub-residual blocks;

[0026] The sub-residual block at any level includes a first convolutional layer, an activation function layer and a second convolutional layer connected in sequence, which is used to process the output results of the sub-residual block of the previous level and the output results of the sub-residual block of the previous level, and output the processing results to the sub-residual block of the next level and the sub-residual block of the next level.

[0027] Optionally, the first discriminator and the second discriminator each include a three-layer discriminant network, and any layer of the discriminant network includes three convolutional layers and one activation function layer.

[0028] Optionally, the bidirectional GAN ​​network model is optimized by the following loss functions: generative adversarial loss function, MSE loss function and total variational loss function.

[0029] A third aspect of the present application discloses a computer device, comprising:

[0030] memory for storing computer programs;

[0031] A processor is configured to implement the steps of the defogging method based on a bidirectional GAN ​​network as described in the first aspect of the present application when executing the computer program.

[0032] In a fourth aspect, the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is processed and executed, the steps of the defogging method based on the bidirectional GAN ​​network as described in the first aspect of the present application are implemented.

[0033] The present application discloses a defogging method based on a bidirectional GAN ​​network and a bidirectional GAN ​​network model. The method includes performing depth processing on the defogging image using the extracted depth information corresponding to different pixels in the defogging image; inputting the depth-processed defogging image into a pre-built bidirectional GAN ​​network model to obtain a defogging image output by the bidirectional GAN ​​network model. In the bidirectional GAN ​​network model, the clear image input port, the first generation unit, and the first discriminator are connected in sequence for feature extraction and reconstruction of the clear image; the fog image input port, the second generation unit, and the second discriminator are connected in sequence for feature extraction and reconstruction of the fog image; the shared latent space stores high-level features extracted by the first encoder for the clear image and high-level features extracted by the second encoder for the fog image; the bidirectional GAN ​​network model is trained and verified through paired clear images and fog images, and includes a bidirectional mapping relationship between the fog domain and the clear domain. It can process images in different domains and effectively ensure the authenticity of the image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 This is a schematic diagram of the structure of the existing dehazing convolutional neural network;

[0036] Figure 2 A schematic diagram of the workflow of a dehazing method based on a bidirectional GAN ​​network disclosed in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the structure of a bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0038] Figure 4 This is a schematic structural diagram of another bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0039] Figure 5 A schematic diagram of a shared latent space in a bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0040] Figure 6 This is a schematic structural diagram of a first generation unit and a second generation unit in a bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0041] Figure 7 This is a schematic structural diagram of a first coupled residual block and a second coupled residual block in a bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0042] Figure 8 This is a schematic diagram of the structure of a first discriminator and a second discriminator in a bidirectional GAN ​​network model disclosed in an embodiment of the present application;

[0043] Figure 9 A schematic diagram of a bidirectional GAN ​​network model for calculating cross-domain conversion consistency disclosed in an embodiment of the present application;

[0044] Figure 10 This is a schematic diagram of an application of a bidirectional GAN ​​network model disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to solve the problem that the current GAN only contains a one-way mapping relationship from the foggy domain to the clear domain, resulting in halo effects and artificial defects in the output of the final image, and poor preservation of the original image information, the present application discloses a defogging method based on a bidirectional GAN ​​network and a bidirectional GAN ​​network model through the following embodiments.

[0046] The first embodiment of this application discloses a defogging method based on a bidirectional GAN ​​network. Figure 2 As shown in the workflow diagram, the defogging method includes:

[0047] Step S11: extracting depth information corresponding to different pixels in the image to be defogged.

[0048] Step S12: performing depth processing on the image to be defogged according to the depth information.

[0049] Specifically, the format of the image to be defogged is converted from RGB format to HSV format based on OpenCV, so as to extract the brightness v and saturation s of the image to be defogged. OpenCV is a cross-platform computer vision and machine learning software library released based on the BSD license (open source). Specifically, in the shadow detection algorithm, the image in RGB format is often converted into HSV format. For the shadow area, its chroma and saturation do not change much compared with the original image, and the brightness information changes greatly. By converting the RGB format into the HSV format, the H, S, and V components can be obtained, thereby obtaining the values ​​of chroma, saturation, and brightness.

[0050] Depth information corresponding to different pixels in the image to be defogged is generated according to the brightness v and the saturation s.

[0051] In actual operation, according to the brightness v and saturation s, the existing depth information formula is used to calculate, and the known parameters θ0, θ1, and θ2 are linearly calculated to obtain the depth information d corresponding to different pixels in the image to be defogged.

[0052] The depth information calculation formula is as follows:

[0053] d(X)=θ0+θ1v(X)+θ2s(X)+ε(X);

[0054] Among them, X represents the image to be dehazed, ε(X) is a random variable representing the random error of the model, and ε can be regarded as a random graph.

[0055] The input original image to be defogged is processed using depth information to enhance the performance of deep areas, such as increasing brightness and contrast in distant areas to make distant scenes clearer.

[0056] In step S13, the image to be dehazed after deep processing is input into a pre-built bidirectional GAN ​​network model.

[0057] join Figure 3 The bidirectional GAN ​​network model includes an input module, a generation module, and a discriminant module. The input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discriminant module includes a first discriminator and a second discriminator. The clear image input port, the first generation unit, and the first discriminator are connected in sequence to perform feature extraction and reconstruction on the clear image, and the fog image input port, the second generation unit, and the second discriminator are connected in sequence to perform feature extraction and reconstruction on the fog image.

[0058] Figure 3The bidirectional GAN ​​network model shown can dehaze foggy images and reversely add fog to clear images, enhancing the consistency of changes across domains. This can better constrain the conversion effect between them, making the dehazed images more natural in specific applications and improving the dehazing effect on real foggy images. However, existing dehazing technologies only use GANs for a one-way conversion from foggy to clear images, outputting only a mapping from the foggy domain to the clear domain. This can cause halo effects and artifacts in the output image, and poorly preserve the original image information.

[0059] join Figure 4 In the bidirectional GAN ​​network model designed in this application, the first generation unit includes a first encoder, a shared latent space, and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space, and a second decoder connected in sequence. The shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder. The high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the foggy image. The training set and validation set of the bidirectional GAN ​​network model both include paired clear images and foggy images.

[0060] In this embodiment, it is assumed that the foggy image and the clear image share a latent space, so the feature connection from the foggy image to the clear image can be completed through the latent space. Figure 5 , the clear image X and the foggy image Y can be connected through a shared latent space Z, restoring the images in both domains. Based on this theoretical foundation, the bidirectional GAN ​​network model disclosed in this application can effectively process both real and synthetic foggy images. It restores and reconstructs foggy images by first transforming the encoding and then decoding them, combined with a bidirectional generative adversarial network.

[0061] Step S14: Obtain the defogging image output by the bidirectional GAN ​​network model.

[0062] The defogging method based on a bidirectional GAN ​​network disclosed in the present application includes performing depth processing on the defogging image using the extracted depth information corresponding to different pixels in the defogging image; inputting the depth-processed defogging image into a pre-built bidirectional GAN ​​network model to obtain a defogging image output by the bidirectional GAN ​​network model. In the bidirectional GAN ​​network model, the clear image input port, the first generation unit, and the first discriminator are connected in sequence for feature extraction and reconstruction of the clear image; the fog image input port, the second generation unit, and the second discriminator are connected in sequence for feature extraction and reconstruction of the fog image; the shared latent space stores high-level features extracted by the first encoder for the clear image and high-level features extracted by the second encoder for the fog image; the bidirectional GAN ​​network model is trained and verified through paired clear images and fog images, and includes a bidirectional mapping relationship between the fog domain and the clear domain. It can process images in different domains and effectively ensure the authenticity of the image reconstruction.

[0063] The second embodiment of the present application discloses a bidirectional GAN ​​network model, combined with Figure 3-Figure 4 The bidirectional GAN ​​network model includes an input module, a generation module and a discrimination module.

[0064] The input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discrimination module includes a first discriminator and a second discriminator.

[0065] The clear image input port, the first generation unit and the first discriminator are connected in sequence for feature extraction and reconstruction of the clear image, and the fog image input port, the second generation unit and the second discriminator are connected in sequence for feature extraction and reconstruction of the fog image.

[0066] The first generation unit includes a first encoder, a shared latent space, and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space, and a second decoder connected in sequence.

[0067] The shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder. The high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the foggy image.

[0068] The training set and validation set of the bidirectional GAN ​​network model both include paired clear images and foggy images.

[0069] For further information, see Figure 6 As shown, the first encoder and the second encoder respectively include a first convolution block, a second convolution block and a first coupling residual block connected in sequence.

[0070] The first decoder and the second decoder respectively include a second coupled residual block, a first stride convolution block and a second stride convolution block connected in sequence.

[0071] An output terminal of the first coupled residual block is connected to an input terminal of the shared latent space, and an output terminal of the shared latent space is connected to an input terminal of the second coupled residual block.

[0072] The output end of the first convolution block is jump-connected to the input end of the second stride convolution block.

[0073] The output end of the second convolution block is jump-connected to the input end of the first-step convolution block.

[0074] Among them, the first convolution block and the second convolution block are used to extract high-level features of the input image, and the first coupled residual block and the second coupled residual block are responsible for learning the detailed information of different features of the image to facilitate the restoration and reconstruction of the image. The first stride convolution and the second stride convolution complete the reconstruction process of the image from high-level features to the output image (clear image or foggy image). The encoder and decoder are closely connected through jump connections. After each layer of convolution block, there will be a jump link connected to the corresponding stride convolution block. The jump connection links the output feature map of the encoder's convolution block to the input of the corresponding decoder and splices it at the dimensional level, so that the feature map output by the decoder can include the feature information of the original image. The jump connection enables the detailed texture of the image to be better learned. Through the connection of two layers of jump connections in high and low dimensions, different position information of the image can be processed separately, further ensuring the consistency of the output image and the input image, as well as the authenticity of the reconstruction.

[0075] For further information, see Figure 7 , the first coupled residual block and the second coupled residual block are respectively composed of a plurality of cascaded sub-residual blocks.

[0076] The sub-residual block at any level includes a first convolutional layer, an activation function layer and a second convolutional layer connected in sequence, which is used to process the output results of the sub-residual block of the previous level and the output results of the sub-residual block of the previous level, and output the processing results to the sub-residual block of the next level and the sub-residual block of the next level.

[0077] The coupled residual block can enhance the network's ability to process information and better fit the potential identity mapping of the deep network. In this embodiment, each sub-residual block consists of two convolutional layers and one activation function layer. Figure 7The outputs and inputs of three sub-residual blocks are cascaded. Each sub-residual block processes its own input and the low-dimensional input from the previous level to extract feature information of different dimensions. The results are then fed into the next sub-residual block and the next-next-level sub-residual block, respectively. By densely connecting these multiple layers of sub-residual blocks, residual information of fog features of different dimensions can be learned.

[0078] After encoding and decoding by the encoder and decoder, the generated image is obtained, and the generated image is input into the discrimination module for discrimination. Figure 8 The first and second discriminators each include a three-layer discriminant network. Each layer of the discriminant network contains three convolutional layers and one activation function. Each layer of the discriminant network downsamples the input of the previous level. The output results of the three-layer discriminant network at different granularities are compared with the actual feature map to determine whether the network output meets expectations. Penalties are imposed on training results that do not meet expectations, and the discriminant loss is calculated and returned to the network to update parameters. A multi-level discriminant network is used to sample and compare the output results of the bidirectional GAN ​​network model through three layers of features, performing coarse-grained and fine-grained comparisons, thereby improving the discrimination accuracy of the discriminant module.

[0079] For the bidirectional GAN ​​network model, this embodiment adopts a loss function, and the auxiliary model can be better expressed. The optimized loss functions include the generative adversarial loss function, the MSE loss function, and the total variational loss function.

[0080] In order to achieve good training results, the network needs to include generative adversarial loss in its loss function. The generative adversarial loss comes from the generator's image G(I) or the real data J, where J is a real image. The classic generative adversarial loss is defined as follows:

[0081] L GAN (G, D) = E I (logD(J))+E I (log(1-D(G(I)));

[0082] Combine Figure 9 , the bidirectional GAN ​​network model calculates the cross-domain conversion consistency by bidirectionally generating adversarial losses as follows:

[0083] L adv =L GAN (G c (I c ), Dis h )+L GAN (G h (I h ), Dis c );

[0084] Among them, LGAN() represents the classic adversarial generation loss, and Ih represents the input fog Figure X Ic represents the input clear image Y, Gc() represents the generative network for clear scenes, Gh() represents the generative network for foggy scenes, Disc represents the discriminative module for clear scenes, and Dish represents the discriminative module for foggy scenes. By combining the shared latent space theory with a bidirectional generative adversarial loss, we can ensure the authenticity of the reconstructed image and the similarity of the results of cross-domain image translation.

[0085] In the training of synthetic fog images, the MSE loss function is used to ensure that the predicted image G(I) is similar to the real image (ground truth) J. The MSE loss function is as follows:

[0086] L MSE =||G c (I c )-J h ||1+||G h (I h )-J c ||1;

[0087] Among them, Jh and Jc represent the fog image data and clear image data in the real scene respectively, and the loss is defined as the difference between the pixels of the generated fog (clear) image and the real fog (clear) image. ||()||1 represents the mean of the sum of the squares of the errors of the corresponding pixels on the image (mean square error).

[0088] The total variation loss function is used in fog image training to eliminate related artificial defects, make the image visually better, and preserve the texture and details of the image. The total variation loss function is as follows:

[0089]

[0090] in, and Represents the horizontal and vertical gradient difference.

[0091] Through the above training, when the model converges to a good accuracy, the obtained training weights can be saved, and then the final bidirectional GAN ​​defogging network model can be obtained. At this time, the model can defog the existing foggy image in an end-to-end manner.

[0092] See also Figure 10 When the bidirectional GAN ​​network model disclosed in this embodiment is deployed in an actual application scenario, the defogging steps are briefly as follows:

[0093] 1. Input a real fog image in a real scene.

[0094] 2. Preprocess the image, convert it from RGB space to HSV space, combine it with known parameters, and output the depth information of the image.

[0095] 3. Combine depth information, process the fog image, decode the fog image and store it in a shared latent space, encode the image in the shared latent space, and generate a clear defogging image in the real scene.

[0096] To enhance the consistency of cross-domain conversion and obtain more realistic dehazed images, the existing unidirectional GAN ​​network was improved. Based on the same shared latent space, the two unidirectional GAN ​​networks (clear image → latent space → foggy image and foggy image → latent space → clear image) were fused into a single module via two sets of encoders and decoders. This formed a bidirectional GAN ​​network model capable of generating foggy / clear images. The cross-domain conversion losses of foggy image → clear image → foggy image and clear image → foggy image → clear image were calculated using the corresponding discriminator, which can better learn the distribution of foggy and clear images.

[0097] The defogging method and bidirectional GAN ​​network model based on the bidirectional GAN ​​network disclosed in this application, based on the shared latent space assumption, proposes a bidirectional defogging network that encodes first and then decodes. This network can process images in different domains, has strong adaptability to cross-domain image conversion, can handle fog information at different depths, and has strong advantages in remote and dense fog processing. The application of jump links, coupled residual blocks, and multi-layer discriminant structures greatly improves the robustness of the network and can more accurately fit the image distribution of the defogging clear image. Real fog information can be used, and depth information can be extracted after preprocessing, which facilitates the processing of fog at deep concentrations and has good expression in real scenes.

[0098] A third embodiment of the present application discloses a computer device, including:

[0099] Memory for storing computer programs.

[0100] A processor, configured to implement the steps of the defogging method based on a bidirectional GAN ​​network as described in the first embodiment of the present application when executing the computer program.

[0101] The fourth embodiment of the present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is processed and executed, the steps of the dehazing method based on the bidirectional GAN ​​network as described in the first embodiment of the present application are implemented.

[0102] The present application has been described in detail above with reference to specific embodiments and exemplary examples. However, these descriptions should not be construed as limiting the present application. Those skilled in the art will appreciate that, without departing from the spirit and scope of the present application, various equivalent substitutions, modifications, or improvements may be made to the technical solutions and implementations of the present application, all of which fall within the scope of the present application. The scope of protection of the present application shall be determined by the appended claims.

Claims

1. A defogging method based on a bidirectional GAN ​​network, characterized in that: The defogging method comprises: Extract the depth information corresponding to different pixels in the image to be defogged; Performing depth processing on the image to be defogged according to the depth information; The image to be defogged after deep processing is input into a pre-built bidirectional GAN ​​network model, wherein the bidirectional GAN ​​network model includes an input module, a generation module and a discriminant module; the input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discriminant module includes a first discriminator and a second discriminator; the clear image input port, the first generation unit and the first discriminator are connected in sequence for extracting and reconstructing features for the clear image, and the fog image input port, the second generation unit and the second discriminator are connected in sequence for extracting and reconstructing features for the fog image; the first generation unit includes a first encoder, a shared latent space and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space and a second decoder connected in sequence; the shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder, the high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the fog image; the training set and the validation set of the bidirectional GAN ​​network model both include paired clear images and fog images; Obtain the dehazed image output by the bidirectional GAN ​​network model.

2. The defogging method based on a bidirectional GAN ​​network according to claim 1 is characterized in that: The extracting depth information corresponding to different pixels in the image to be defogged includes: Performing format conversion on the image to be defogged, and extracting the brightness and saturation of the image to be defogged; Depth information corresponding to different pixels in the image to be defogged is generated according to the brightness and saturation.

3. A defogging method based on a bidirectional GAN ​​network, wherein the defogging method is applied to a bidirectional GAN ​​network model, characterized in that: The bidirectional GAN ​​network model includes an input module, a generation module and a discrimination module; The input module includes a clear image input port and a fog image input port, the generation module includes a first generation unit and a second generation unit, and the discrimination module includes a first discriminator and a second discriminator; The clear image input port, the first generation unit, and the first discriminator are connected in sequence for extracting and reconstructing features for the clear image; the fog image input port, the second generation unit, and the second discriminator are connected in sequence for extracting and reconstructing features for the fog image; The first generation unit includes a first encoder, a shared latent space, and a first decoder connected in sequence, and the second generation unit includes a second encoder, the shared latent space, and a second decoder connected in sequence; The shared latent space is used to store high-level features and output the high-level features to the first decoder and the second decoder, wherein the high-level features include the high-level features extracted by the first encoder for the clear image and the high-level features extracted by the second encoder for the foggy image; The training set and validation set of the bidirectional GAN ​​network model both include paired clear images and foggy images.

4. The defogging method based on a bidirectional GAN ​​network according to claim 3 is characterized in that: The first encoder and the second encoder respectively include a first convolution block, a second convolution block and a first coupled residual block connected in sequence; The first decoder and the second decoder respectively include a second coupled residual block, a first stride convolution block and a second stride convolution block connected in sequence; An output terminal of the first coupled residual block is connected to an input terminal of the shared latent space, and an output terminal of the shared latent space is connected to an input terminal of the second coupled residual block; The output end of the first convolution block is jump-connected to the input end of the second stride convolution block; The output end of the second convolution block is jump-connected to the input end of the first-step convolution block.

5. The defogging method based on a bidirectional GAN ​​network according to claim 4 is characterized in that: The first coupled residual block and the second coupled residual block are respectively composed of a plurality of cascaded sub-residual blocks; The sub-residual block at any level includes a first convolutional layer, an activation function layer and a second convolutional layer connected in sequence, which is used to process the output results of the sub-residual block of the previous level and the output results of the sub-residual block of the previous level, and output the processing results to the sub-residual block of the next level and the sub-residual block of the next level.

6. The defogging method based on a bidirectional GAN ​​network according to claim 3 is characterized in that: The first discriminator and the second discriminator both include a three-layer discriminant network and a one-layer granularity discriminant network, and any layer of the discriminant network includes three convolutional layers and one activation function layer.

7. The defogging method based on a bidirectional GAN ​​network according to claim 3 is characterized in that: The bidirectional GAN ​​network model is optimized through the following loss functions: generative adversarial loss function, MSE loss function and total variation loss function.

8. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the defogging method based on a bidirectional GAN ​​network as claimed in claim 1 or 2 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is processed and executed, the steps of the defogging method based on the bidirectional GAN ​​network are implemented as claimed in claim 1 or 2.

Images