Network training, application method and device based on spatial transformation and contrast learning

Abstract

The application provides a network training, application method and equipment based on spatial transformation and contrast learning, the training method introduces an initial spatial transformation network, an initial atmospheric light map generator and an initial transmission map generator, and combines processing sequence consistency constraints and bidirectional contrast loss to jointly train the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator and the initial transmission map generator, and obtain a target dehazing network corresponding to the initial dehazing network. Through the processing sequence consistency constraints, the adaptability and anti-interference ability of the target dehazing network to image geometric disturbance can be enhanced, the robustness of the target dehazing network is enhanced, and the generalization ability and dehazing stability of the target dehazing network in a complex scene are significantly improved. Through the bidirectional contrast loss, learning confusion caused by data distribution difference can be avoided, the constraint ability on clear images is enhanced, and the image reconstruction quality of the target dehazing network is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision image enhancement technology, and in particular to a network training and application method and device based on spatial transformation and contrastive learning. Background Technology

[0002] In recent years, with the decline in air quality and the reduction in visibility, severe weather conditions (such as smog, dense fog, and dust) have brought many challenges to people's daily lives and industrial production.

[0003] Due to atmospheric particle scattering and ambient light interference, images taken outdoors often suffer from reduced contrast, dull colors, and brightness shifts, severely impacting the recognition and analysis capabilities of computer vision systems. Therefore, effectively removing haze from images and improving visual clarity has become a crucial issue that urgently needs to be addressed. Summary of the Invention

[0004] This invention provides a network training and application method and device based on spatial transformation and contrastive learning to overcome the deficiencies in the prior art.

[0005] This invention provides a network training method based on spatial transformation and contrastive learning, comprising:

[0006] Based on the initial spatial transformation network and the initial dehazing network, the first deformed image of the foggy image in the sample set and the first clear image of the first deformed image are determined successively. Based on the initial dehazing network and the initial spatial transformation network, the second clear image of the foggy image and the second deformed image of the second clear image are determined successively.

[0007] Based on the initial atmospheric light map generator, the initial atmospheric light map of the sample foggy image is determined, and based on the initial transmission map generator, the initial transmission map of the sample foggy image is determined.

[0008] Based on the second clear image, the initial atmospheric light map, and the initial transmission map, a foggy image is reconstructed to obtain a first reconstruction result. Based on the sample foggy image, the initial atmospheric light map, and the initial transmission map, a clear image is reconstructed to obtain a second reconstruction result.

[0009] Based on the first clear image and the second deformed image, the processing order consistency constraint is calculated, and based on the sample foggy image, the second clear image, the first reconstruction result, and the second reconstruction result, the bidirectional contrast loss is calculated.

[0010] Based on the processing order consistency constraint and the bidirectional contrast loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network corresponding to the initial dehazing network.

[0011] According to the present invention, a network training method based on spatial transformation and contrastive learning is provided, wherein the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained based on the processing order consistency constraint and the bidirectional contrastive loss to obtain a target dehazing network corresponding to the initial dehazing network, comprising:

[0012] Based on the initial discriminator, the second clear image and the second reconstruction result are discriminated separately to obtain the first discrimination result and the second discrimination result;

[0013] Based on the first discrimination result and the second discrimination result, calculate the adversarial loss;

[0014] Based on the adversarial loss, the processing order consistency constraint, and the bidirectional contrast loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network.

[0015] According to the present invention, a network training method based on spatial transformation and contrastive learning, wherein the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained based on the processing order consistency constraint and the bidirectional contrastive loss to obtain the target dehazing network corresponding to the initial dehazing network, further comprising:

[0016] Based on the first reconstruction result and the sample foggy image, calculate the content consistency loss; based on the second reconstruction result and the second clear image, calculate the clarity consistency loss; and based on the second reconstruction result and the second clear image, calculate the structural similarity loss.

[0017] Based on the processing order consistency constraint, the bidirectional contrast loss, and at least one of the content consistency loss, the clarity consistency loss, and the structural similarity loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network.

[0018] According to the present invention, a network training method based on spatial transformation and contrastive learning is provided, wherein the initial dehazing network includes a U-Net architecture, and the U-Net architecture includes an input convolutional layer, multiple downsampling layers, multiple upsampling layers, and an output convolutional layer;

[0019] Hybrid structure blocks are provided between the input convolutional layer and the first downsampling layer, between two adjacent downsampling layers, between two adjacent upsampling layers, between the last upsampling layer and the output convolutional layer, and after the output convolutional layer; the hybrid structure block includes a multi-scale parallel convolutional kernel module and a parallel attention module.

[0020] According to a network training method based on spatial transformation and contrastive learning provided by the present invention, the initial spatial transformation network includes a localization network, a gradient flipping layer, a network generator, and a sampler connected in sequence.

[0021] The positioning network is used to generate the spatial transformation parameters of the input image of the initial spatial transformation network;

[0022] The gradient flipping layer is used to invert gradients during backpropagation in the joint training process;

[0023] The network generator is used to determine the coordinate grid corresponding to the output image of the initial spatial transformation network based on the spatial transformation parameters; the coordinate grid corresponds to the position of each pixel in the input image of the initial spatial transformation network.

[0024] The sampler is used to sample the input image of the initial spatial transformation network based on the coordinate grid to obtain the output image of the initial spatial transformation network.

[0025] According to a network training method based on spatial transformation and contrastive learning provided by the present invention, the initial atmospheric light map generator includes an encoder, an intermediate block and a decoder connected in sequence;

[0026] The encoder is used to extract features from the sample foggy image;

[0027] The intermediate block is used to convert the features extracted by the encoder into the mean and variance of a Gaussian distribution, and to perform sampling based on parameterization techniques to obtain the sampling results;

[0028] The decoder is used to decode the sampling results to obtain the initial atmospheric light map.

[0029] According to the present invention, a network training method based on spatial transformation and contrastive learning is provided, wherein the initial transmission graph generator includes a first downsampling layer, a first standard convolutional block, a first upsampling layer, a second downsampling layer, a first splicing layer, a second standard convolutional block, a second upsampling layer, a third downsampling layer, a second splicing layer, a third standard convolutional block, a third splicing layer, and a fourth standard convolutional block;

[0030] The first downsampling layer, the first standard convolutional block, and the first upsampling layer are connected in sequence; the second downsampling layer, the first splicing layer, the second standard convolutional block, and the second upsampling layer are connected in sequence; and the first upsampling layer is connected to the first splicing layer.

[0031] The third downsampling layer, the second splicing layer, the third standard convolutional block, the third splicing layer, and the fourth standard convolutional block are connected in sequence, and the second upsampling layer is connected to the second splicing layer;

[0032] The first downsampling layer, the second downsampling layer, the third downsampling layer, and the third stitching layer are respectively connected to the sample foggy image, and the fourth standard convolution outputs the initial transmission image.

[0033] This invention also provides a network application method based on spatial transformation and contrastive learning, comprising:

[0034] Obtain the image to be dehazed;

[0035] The image to be dehazed is input into the target dehazing network to obtain the target clear image output by the target dehazing network;

[0036] The target dehazing network is trained based on the aforementioned network training method based on spatial transformation and contrastive learning.

[0037] The present invention also provides an apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a network training method based on spatial transformation and contrastive learning as described above, or a network application method based on spatial transformation and contrastive learning.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network training method based on spatial transformation and contrastive learning, or the network application method based on spatial transformation and contrastive learning, as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a network training method based on spatial transformation and contrastive learning, or a network application method based on spatial transformation and contrastive learning, as described above.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention provides a network training and application method and device based on spatial transformation and contrastive learning. The training method introduces an initial spatial transformation network, an initial atmospheric light map generator, and an initial transmission map generator. It combines processing order consistency constraints and bidirectional contrastive loss to jointly train these three networks, resulting in a target dehazing network corresponding to the initial dehazing network. By processing order consistency constraints, the target dehazing network's adaptability and anti-interference ability to image geometric perturbations (such as rotation and scaling) are enhanced, improving its robustness and significantly increasing its generalization ability and dehazing stability in complex scenarios such as camera deflection, UAV tilt, and changes in remote sensing angle. The bidirectional contrastive loss avoids learning confusion caused by differences in data distribution while enhancing the constraint on clear images, thus improving the image reconstruction quality of the target dehazing network. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on the drawings described below without creative effort.

[0043] Figure 1 This is one of the flowcharts of the network training method based on spatial transformation and contrastive learning provided by the present invention;

[0044] Figure 2 This is a schematic diagram of the U-Net architecture, which serves as the initial defogging network in the network training method based on spatial transformation and contrastive learning provided by this invention.

[0045] Figure 3 This is a schematic diagram of the multi-scale parallel convolutional kernel module of the U-Net architecture, which serves as the initial defogging network in the network training method based on spatial transformation and contrastive learning provided by this invention.

[0046] Figure 4 This is a schematic diagram of the parallel attention module of the U-Net architecture, which serves as the initial dehazing network in the network training method based on spatial transformation and contrastive learning provided by this invention.

[0047] Figure 5 This is a schematic diagram of the structure of the initial atmospheric light map generator in the network training method based on spatial transformation and contrastive learning provided by the present invention;

[0048] Figure 6 This is a schematic diagram of the structure of the initial transmission graph generator in the network training method based on spatial transformation and contrastive learning provided by the present invention;

[0049] Figure 7 This is a complete flowchart of the network training method based on spatial transformation and contrastive learning provided by the present invention;

[0050] Figure 8 This is a flowchart illustrating the network application method based on spatial transformation and contrastive learning provided by the present invention.

[0051] Figure 9 This is a schematic diagram of the structure of the network training device based on spatial transformation and contrastive learning provided by the present invention;

[0052] Figure 10 This is a schematic diagram of the structure of the network application device based on spatial transformation and contrastive learning provided by the present invention;

[0053] Figure 11 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] Figure 1 This is a flowchart illustrating a network training method based on spatial transformation and contrastive learning provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0056] S11, based on the initial spatial transformation network and the initial dehazing network, the first deformed image of the sample hazy image in the sample set and the first clear image of the first deformed image are determined successively, and based on the initial dehazing network and the initial spatial transformation network, the second clear image of the sample hazy image and the second deformed image of the second clear image are determined successively.

[0057] S12, Based on the initial atmospheric light map generator, determine the initial atmospheric light map of the sample foggy image, and based on the initial transmission map generator, determine the initial transmission map of the sample foggy image.

[0058] S13, based on the second clear image, the initial atmospheric light map and the initial transmission map, reconstruct the foggy image to obtain the first reconstruction result, and based on the sample foggy image, the initial atmospheric light map and the initial transmission map, reconstruct the clear image to obtain the second reconstruction result;

[0059] S14, based on the first clear image and the second deformed image, calculate the processing order consistency constraint, and based on the sample foggy image, the second clear image, the first reconstruction result and the second reconstruction result, calculate the bidirectional contrast loss;

[0060] S15, based on the processing order consistency constraint and the bidirectional contrast loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network corresponding to the initial dehazing network.

[0061] Specifically, the network training method based on spatial transformation and contrastive learning provided in this embodiment of the invention is executed by a network training device based on spatial transformation and contrastive learning. This device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0062] First, step S11 is executed to obtain a sample set for training the initial dehazing network to obtain the target dehazing network. This sample set includes multiple foggy images and clear images of samples that are not paired with each foggy image.

[0063] The hazy sample images are processed in two sequences. The first sequence involves inputting the hazy sample image into an initial Spatial Transformer Network (STN) to obtain a first deformed image of the hazy sample image output by the STN. Then, this first deformed image is input into an initial dehazing network to obtain a first clear image of the first deformed image output by the initial dehazing network.

[0064] The second processing sequence is to first input the hazy sample image into the initial dehazing network to obtain the second clear image of the hazy sample image output by the initial dehazing network, and then input the second clear image into the initial spatial transformation network to obtain the second deformed image of the second clear image output by the initial spatial transformation network.

[0065] The initial spatial transformation network is a rotational deformation network, which may include a positioning network, a network generator, and a sampler.

[0066] The design of the rotation deformation network allows for a wide range of deformation of the input image to the initial spatial transformation network in order to continuously update the deformation parameters. Specifically, the localization network processes the input image of the initial spatial transformation network to generate the input image of the initial spatial transformation network. Spatial transformation parameters Used for deformation, i.e. .in, H represents the location network. The height, W is The width, C is The number of channels.

[0067] Network Generator Used for Determine the initial spatial transformation network Output image The corresponding coordinate grid, i.e., establishing deformation and The corresponding coordinate relationship between them express height, for The width. There are:

[0068] ;

[0069] in, express The coordinates of the i-th pixel. express The i-th pixel in The corresponding sampling coordinates in yes The j-th element in.

[0070] Through the sampler By sampling the pixel values ​​in the image, you can obtain... .

[0071] The sampling process of the sampler can be described as follows:

[0072] ;

[0073] in, for The pixel value of the i-th pixel in channel c. , . express exist The pixel value at the specified location. For the sampling function, bilinear interpolation can be used here. , and All The parameters. Therefore, the above formula can be simplified to:

[0074] ;

[0075] in, express exist The weight coefficient of the pixel at the specified location.

[0076] It is understandable that the sample image with fog is used as... hour, The first deformed image represents the sample image with fog, while the second clear image is used as... hour, The second deformed image represents the second clear image.

[0077] The initial dehazing network is an end-to-end dehazing network (Parallel Attention Net, PANet), which can be a U-Net architecture or other network architectures; no specific limitation is made here.

[0078] Then, step S12 is executed, introducing an initial atmospheric illumination map generator (A-Net) and an initial transmission map generator (T-Net). The initial atmospheric illumination map generator is used to estimate the initial atmospheric illumination map of the sample foggy image. The initial transmission map generator is used to estimate the initial transmission map of the sample foggy image. The initial transmission map reflects the proportion of light penetrating the fog layer in the scene and is an important intermediate variable for recovering a clear image.

[0079] Inputting a sample hazy image into an initial atmospheric light map generator will produce an initial atmospheric light map of the sample hazy image output by the initial atmospheric light map generator. Inputting a sample hazy image into an initial transmission map generator will produce an initial transmission map of the sample hazy image output by the initial transmission map generator.

[0080] The initial atmospheric light map generator and the initial transport map generator can also be U-Net architectures or other network architectures. Adaptive Fine-Grained Channel Attention (FCA) or other structures can be added to the decoder of the U-Net architecture. For example, the initial atmospheric light map generator can also employ a Variational Auto-Encoder (VAE) structure, which can consist of a Convolutional Neural Network (CNN)-based encoder, a symmetric decoder, and intermediate blocks. The initial transport map generator can also employ a multi-scale structure, achieving information fusion at different scales through progressive downsampling and upsampling to improve the effectiveness and robustness of dehazing.

[0081] Then, step S13 is executed, using the second clear image, the initial atmospheric light map and the initial transmission map, combined with the atmospheric scattering model, to reconstruct the foggy image and obtain the first reconstruction result.

[0082] The atmospheric scattering model can be expressed by the following formula:

[0083] ;

[0084] in, For images with fog, A clear image of a foggy image. This is a transmission graph of a foggy image. This is an atmospheric light map of a foggy image, where 'a' represents a pixel.

[0085] The first reconstruction result can be expressed as:

[0086] ;

[0087] in, J represents the first reconstruction result, A represents the second clear image, T represents the initial atmospheric light map, and T represents the initial transmission map.

[0088] By using sample foggy images, initial atmospheric light maps, and initial transmission maps, combined with an atmospheric scattering model, a clear image can be reconstructed, resulting in a second reconstruction result.

[0089] The second reconstruction result can be expressed as:

[0090] ;

[0091] in, The second reconstruction result is shown in Figure I, where I represents a sample image with fog.

[0092] Next, step S14 is executed, using the first clear image and the second deformed image to calculate the processing order consistency constraint. This processing order consistency constraint is an adversarial rotation consistency constraint, which alternately optimizes the initial dehazing network and the initial spatial transformation network when only a foggy image is available. This makes the initial dehazing network insensitive to deformation operations, thereby achieving better dehazing results and robustness. For the initial dehazing network, the consistency of the dehazing result must be guaranteed regardless of the order of the dehazing and deformation operations. In contrast, the initial spatial transformation network exhibits an adversarial tendency, that is, maximizing the dissimilarity between different operation sequences.

[0093] The processing order consistency constraint can be expressed as:

[0094] ;

[0095] in, This represents the minimum loss of the initial dehazing network. To maximize the loss of the initial spatial transformation network, Indicates a sample image with fog Follows fog map distribution Fog map distribution It is the distribution that all samples in the sample set with fog images follow. This represents the expectation that all samples in the sample set have foggy images, which can ensure generalization. For the initial defogging network, For the second clearest image, The first deformed image, For the first clear image, For the second deformed image, Used to measure the difference between the results of two processing orders.

[0096] We can also utilize the sample hazy image I, the second clear image J, and the first reconstruction result. Second reconstruction results Calculate the bidirectional contrast loss. The bidirectional contrast loss can include dehazing contrast constraints and foggy contrast constraints.

[0097] The dehazing contrast constraint uses the second clear image J as the anchor point and the second reconstruction result as the basis. Positive sample, first reconstruction result The image I with fog is constructed as a negative sample, as detailed below:

[0098] ;

[0099] With fog contrast constraints, the first reconstruction result is used. Using the hazy image I as the anchor point, and the second clear image J and the second reconstruction result as the positive samples, the image is used as the anchor point. The negative samples are constructed as follows:

[0100] ;

[0101] in, For defogging contrast constraints, For the sake of fog contrast constraint, These represent extractions from a fixed pre-trained model of VGG19. The i-th hidden feature. It is a distance function used to calculate distance, It is the weight coefficient corresponding to the i-th hidden feature.

[0102] The bidirectional contrast loss is shown below:

[0103] ;

[0104] in, and The hyperparameters are used to balance the loss.

[0105] By constructing a bidirectional contrast loss, constraints on image sharpness and content can be effectively compensated, improving the initial dehazing network's ability to learn image features and enhancing overall dehazing performance. Furthermore, the bidirectional contrast loss can strengthen the constraints of the initial dehazing network on image reconstruction, achieving even better dehazing results.

[0106] Here, by handling the sequential consistency constraint and the bidirectional contrast loss, an unsupervised triple contrast loss can be formed, which can realize contrastive learning.

[0107] Finally, step S15 is executed, where a weighted sum of the processing order consistency constraint and the bidirectional contrast loss is calculated to obtain the training loss. Then, using the training loss, backpropagation is performed to jointly train the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator in unsupervised mode, resulting in the target dehazing network corresponding to the initial dehazing network.

[0108] The network training method based on spatial transformation and contrastive learning provided in this embodiment of the invention introduces an initial spatial transformation network, an initial atmospheric light map generator, and an initial transmission map generator. Combined with processing order consistency constraints and bidirectional contrastive loss, the initial spatial transformation network, initial dehazing network, initial atmospheric light map generator, and initial transmission map generator are jointly trained to obtain the target dehazing network corresponding to the initial dehazing network. By processing order consistency constraints, the adaptability and anti-interference ability of the target dehazing network to image geometric perturbations (such as rotation, scaling, etc.) can be enhanced, improving the robustness of the target dehazing network and significantly improving its generalization ability and dehazing stability in complex scenarios such as camera deflection, UAV tilt, and changes in remote sensing angle. The bidirectional contrastive loss avoids learning confusion caused by differences in data distribution and enhances the constraint ability on clear images, improving the image reconstruction quality of the target dehazing network. This training method, combining processing order consistency constraints and bidirectional contrastive loss, forms an unsupervised triple contrastive loss, enabling the obtained target dehazing network to effectively remove haze from images and improve visual clarity.

[0109] Based on the above embodiments, the joint training of the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator based on the processing order consistency constraint and the bidirectional contrast loss to obtain the target dehazing network corresponding to the initial dehazing network includes:

[0110] Based on the initial discriminator, the second clear image and the second reconstruction result are discriminated separately to obtain the first discrimination result and the second discrimination result;

[0111] Based on the first discrimination result and the second discrimination result, calculate the adversarial loss;

[0112] Based on the adversarial loss, the processing order consistency constraint, and the bidirectional contrast loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network.

[0113] Specifically, when calculating the training loss, adversarial loss can also be calculated. When calculating adversarial loss, the initial discriminator can be used to distinguish between the second clear image and the second reconstruction result, respectively, to obtain the first discrimination result and the second discrimination result.

[0114] The second clear image, along with clear images of samples not paired with the foggy images in the sample set, are input into the initial discriminator. The initial discriminator then outputs a first discrimination result. This first discrimination result represents the probability that the second clear image is real data, indicating the likelihood that the second clear image is real data.

[0115] The second reconstruction result, along with clear images of samples from the sample set that were not paired with the foggy images, are input into the initial discriminator. The initial discriminator then outputs the second discrimination result. This second discrimination result represents the probability that the second reconstruction result is real data, indicating the likelihood that the second reconstruction result is real data.

[0116] Subsequently, the adversarial loss is calculated using the first and second discrimination results.

[0117] Adversarial loss can be described as solving an extreme optimization problem, and can include the first adversarial loss corresponding to the first discrimination result and the second adversarial loss corresponding to the second discrimination result:

[0118] ;

[0119] ;

[0120] ;

[0121] in, As the first instance of combat losses, For the second confrontation loss, This represents the loss of the initial defogging network. This represents the loss in reconstructing a clear image using an atmospheric scattering model. Indicates the initial discriminator The loss, This indicates a hazy image of the sample. This represents the distribution that all fog-free images in the sample set follow.

[0122] Subsequently, the training loss is calculated using processing order consistency constraints and bidirectional contrastive loss, combined with adversarial loss, and backpropagation is performed to jointly train the initial discriminator, initial spatial transformation network, initial dehazing network, initial atmospheric light map generator, and initial transport map generator.

[0123] In this embodiment of the invention, introducing adversarial loss enables the initial dehazing network to generate realistic images, while allowing the initial discriminator to better distinguish between real and generated samples. Introducing adversarial loss makes the training process more stable, reduces mode collapse, improves the performance of the target dehazing network, and ensures that the dehazed image obtained by the target dehazing network has both global consistency and preserves local details, reducing color deviation and blurring issues.

[0124] Based on the above embodiments, the step of jointly training the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator based on the processing order consistency constraint and the bidirectional contrast loss to obtain the target dehazing network corresponding to the initial dehazing network further includes:

[0125] Based on the first reconstruction result and the sample foggy image, calculate the content consistency loss; based on the second reconstruction result and the second clear image, calculate the clarity consistency loss; and based on the second reconstruction result and the second clear image, calculate the structural similarity loss.

[0126] Based on the processing order consistency constraint, the bidirectional contrast loss, and at least one of the content consistency loss, the clarity consistency loss, and the structural similarity loss, the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator are jointly trained to obtain the target dehazing network.

[0127] Specifically, when calculating the training loss, at least one of the following can also be calculated: content consistency loss, clarity consistency loss, and structural similarity loss.

[0128] The content consistency loss can be calculated using the first reconstruction result and the sample hazy image. The content consistency loss can be expressed as:

[0129] ;

[0130] in, For loss of content consistency, This is the first reconstruction result. It is an L1 norm.

[0131] The sharpness consistency loss can be calculated using the second reconstruction result and the second sharp image. The sharpness consistency loss can be expressed as:

[0132] ;

[0133] in, To clarify the loss of consistency, This is the result of the second reconstruction.

[0134] The structural similarity loss can be calculated using the second reconstruction result and the second clear image. The structural similarity loss can be expressed as:

[0135] ;

[0136] in, It is the mean of the second clearest image. It is the mean of the second reconstruction results. This represents the variance of the second clearest image. This represents the variance of the second reconstruction result. This represents the covariance between the second clear image and the second reconstruction result. and There are two variables during the training process. Take as , Take as , It refers to the dynamic range of pixel values, from 0 to 255. Set it to 0.01. The value is set to 0.03.

[0137] Furthermore, the training loss is calculated by using processing order consistency constraints and bidirectional contrast loss, combined with at least one of content consistency loss, clarity consistency loss and structural similarity loss, and backpropagation is performed to jointly train the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator and the initial transport map generator.

[0138] In this embodiment of the invention, the introduction of content consistency loss ensures content consistency between the second clear image output by the initial dehazing network and the input hazy sample image. The introduction of clarity consistency loss further enhances the constraint of the initial dehazing network on the reconstructed clear image, thereby improving the dehazing effect of the initial dehazing network. The introduction of structural similarity loss enhances the structural similarity between the second reconstruction result and the second clear image.

[0139] Based on the above embodiments, atmospheric light map estimation loss can also be calculated, and training loss can be calculated by using atmospheric light map estimation loss, processing order consistency constraint and bidirectional contrast loss, combined with at least one of content consistency loss, clarity consistency loss and structural similarity loss, and backpropagation can be performed to jointly train the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator and the initial transport map generator.

[0140] The atmospheric light map estimation loss can be expressed as:

[0141] ;

[0142] in, This represents the loss between the initial atmospheric light map and the actual atmospheric light map. The KL (Kullback-Leibler) divergence loss is used.

[0143] It can be represented as:

[0144] ;

[0145] in, To estimate the loss for atmospheric light maps, This is the initial atmospheric light map. To obtain a true atmospheric light map, the Dark Channel Prior (DCP) can be calculated from the sample image with fog. This represents the mean square error loss.

[0146] The calculation can be performed using the following steps:

[0147] (1) Determine the dark channel value of each pixel in the hazy image of the sample. That is: ;

[0148] in, Let be the dark channel value of pixel i in the sample hazy image. Let c be a local image patch centered at pixel i in the hazy sample image, i.e., a small patch around pixel i. Let c be the color channel, r be the red channel, g be the green channel, and b be the blue channel. Let y be the pixel value of pixel y in color channel c of the sample hazy image. The pixels in the image.

[0149] (2) Using the dark channel values, the true atmospheric light map corresponding to the sample foggy image can be determined, namely: ;

[0150] in, This is the actual atmospheric light map corresponding to the sample image with fog. This is a set of a preset number of pixels with high dark channel values. is the pixel value of pixel j in the real atmospheric light image.

[0151] Latent variables can be obtained by encoding sample foggy images using an initial atmospheric light map generator. The normal distribution and the standard normal distribution are calculated to force latent variables to conform to the standard normal distribution. The mean of the latent variables. The variance is the latent variable.

[0152] It can be calculated using the following formula:

[0153] ;

[0154] Where KL represents KL divergence, which measures the difference between two probability distributions, and || represents the calculation of KL divergence between two probability distributions. Let represent the i-th dimension of the latent variable z.

[0155] In this embodiment of the invention, KL divergence loss is introduced, and the standard stochastic gradient method can be used for end-to-end model training, thereby improving the performance of the initial atmospheric light map generator in generating atmospheric light maps.

[0156] In summary, the training loss used in the embodiments of the present invention can be calculated using the following formula:

[0157] ;

[0158] in, These are the hyperparameters that balance different losses. .

[0159] Based on the above embodiments, the initial dehazing network includes a U-Net architecture, which includes an input convolutional layer, multiple downsampling layers, multiple upsampling layers, and an output convolutional layer.

[0160] Hybrid structure blocks are provided between the input convolutional layer and the first downsampling layer, between two adjacent downsampling layers, between two adjacent upsampling layers, between the last upsampling layer and the output convolutional layer, and after the output convolutional layer; the hybrid structure block includes a multi-scale parallel convolutional kernel module and a parallel attention module.

[0161] Specifically, such as Figure 2 As shown, the U-Net architecture includes an input convolutional (conv) layer, multiple downsampling (downsample) layers, multiple upsampling (upsample) layers, and an output convolutional layer. The input convolutional layer is used to input I, and the output convolutional layer is used to output J.

[0162] Mix structure blocks are set between the input convolutional layer and the first downsampling layer, between two adjacent downsampling layers, between two adjacent upsampling layers, between the last upsampling layer and the output convolutional layer, and after the output convolutional layer.

[0163] The hybrid structure block includes a connected multi-scale parallel convolutional kernel module (MSPLCK) and a parallel attention module. The multi-scale parallel convolutional kernel module is as follows: Figure 3As shown, it includes a batch normalization layer (Batch Norm), a 1×1 convolutional layer (1×1conv), a 5×5 convolutional layer (5×5conv), a pointwise convolutional layer with a dilated kernel size of 7 (DWDConv7), a pointwise convolutional layer with a dilated kernel size of 13 (DWDConv13), a pointwise convolutional layer with a dilated kernel size of 19 (DWDConv19), a splicing layer (C), a 1×1 convolutional layer (1×1conv), an activation layer (GELU), a 1×1 convolutional layer (1×1conv), and an addition layer (+).

[0164] Parallel attention modules, such as Figure 4 As shown, the initial dehazing network includes a batch normalization layer, a spatial attention module, a pixel attention module, a channel attention module, a stitching layer (C), a 1×1 convolutional layer (1×1conv), an activation layer (GELU), another 1×1 convolutional layer (1×1conv), and an additive layer (+). By combining spatial attention, pixel attention, and channel attention mechanisms, the initial dehazing network can more accurately extract and analyze key information in the image, thereby improving the computational accuracy and reliability of the dehazing task.

[0165] In this embodiment of the invention, multi-scale parallel convolutional kernels and parallel attention mechanisms are combined, which can more effectively capture local texture information, enhance the ability of the target dehazing network to restore details during the dehazing process, and adapt to the dehazing needs of different regions, thereby generating higher quality images.

[0166] Based on the above embodiments, the initial spatial transformation network includes a localization network, a gradient flipping layer, a network generator, and a sampler connected in sequence;

[0167] The positioning network is used to generate the spatial transformation parameters of the input image of the initial spatial transformation network;

[0168] The gradient flipping layer is used to invert gradients during backpropagation in the joint training process;

[0169] The network generator is used to determine the coordinate grid corresponding to the output image of the initial spatial transformation network based on the spatial transformation parameters; the coordinate grid corresponds to the position of each pixel in the input image of the initial spatial transformation network.

[0170] The sampler is used to sample the input image of the initial spatial transformation network based on the coordinate grid to obtain the output image of the initial spatial transformation network.

[0171] Specifically, the initial spatial transformation network comprises a localization network, a gradient flip layer (GRL), a network generator, and a sampler connected in sequence. The gradient flip layer enables the initial spatial transformation network and the initial dehazing network to undergo adversarial training by inverting gradients, further enhancing the robustness of the target dehazing network to image geometric perturbations and improving its generalization ability and dehazing stability in complex scenes.

[0172] Based on the above embodiments, the initial atmospheric light map generator includes an encoder, an intermediate block, and a decoder connected in sequence;

[0173] The encoder is used to extract features from the sample foggy image;

[0174] The intermediate block is used to convert the features extracted by the encoder into the mean and variance of a Gaussian distribution, and to perform sampling based on parameterization techniques to obtain the sampling results;

[0175] The decoder is used to decode the sampling results to obtain the initial atmospheric light map.

[0176] Specifically, the initial atmospheric light map generator comprises an encoder, intermediate blocks, and a decoder connected in sequence. The encoder consists of multiple sequentially connected Class I convolutional blocks used to extract features from the sample foggy image. The intermediate blocks (using the reparemeterization trick) are used to transform the features extracted by the encoder into the mean of a Gaussian distribution. and variance And utilize parameterization techniques Sampling was performed to obtain the sampling results. This ensures gradient propagation, allowing for end-to-end optimization of the initial atmospheric light map generator. The decoder consists of multiple convolutional blocks of type II connected in sequence, and a convolutional block of type III connected to the last convolutional block of type II.

[0177] like Figure 5 As shown, the encoder includes four first-class convolutional blocks, and the decoder includes four second-class convolutional blocks and one third-class convolutional block. The first-class convolutional blocks may include a 3×3 convolutional layer, an activation layer (ReLU), and a max pooling layer connected in sequence. The second-class convolutional blocks include an upsampling layer, a 3×3 convolutional layer, a batch normalization layer, and an activation layer (ReLU). The third-class convolutional block includes an upsampling layer, a 5×5 convolutional layer, and an activation layer (Sigmoid).

[0178] Based on the above embodiments, such as Figure 6As shown, the initial transmission graph generator includes a first downsampling layer, a first standard convolutional block (G4), a first upsampling layer, a second downsampling layer, a first stitching layer, a second standard convolutional block (G3), a second upsampling layer, a third downsampling layer, a second stitching layer, a third standard convolutional block (G2), a third stitching layer, and a fourth standard convolutional block (G1).

[0179] The first downsampling layer, G4, and the first upsampling layer are connected in sequence. The second downsampling layer, the first stitching layer, G3, and the second upsampling layer are connected in sequence. The first upsampling layer is connected to the first stitching layer.

[0180] The third downsampling layer, the second stitching layer, G2, the third stitching layer, and G1 are connected in sequence, and the second upsampling layer is connected to the second stitching layer.

[0181] The first downsampling layer, the second downsampling layer, the third downsampling layer, and the third stitching layer are respectively connected to the sample foggy image, and G1 outputs the initial transmission image.

[0182] Specifically, the inputs to the first, second, and third downsampling layers are all hazy sample images. The image features are progressively compressed to lower resolution to extract deeper haze features, and the outputs are respectively... The scales are respectively At each scale, introduce For feature learning, each standard convolutional block includes a convolutional layer (Conv), a batch normalization layer (BN), and Leaky ReLU (LReLU), which is... The network structure is optimized to ensure stable gradient propagation and improve feature representation capabilities. Simultaneously, skip connections are designed to integrate different scales. Direct connection to the corresponding decoding stage is used to preserve more local detail and mitigate information loss caused by multiple downsampling steps. Subsequently, the network restores the feature map resolution through progressive upsampling, enabling effective fusion of low-scale features at a high-resolution level, ultimately culminating in the highest-resolution fourth standard convolutional block. The initial transmission map is output here for subsequent defogging tasks.

[0183] In this embodiment of the invention, a multi-scale initial transmission map generator is introduced. By capturing global and local features at different scales, the accuracy, robustness, and efficiency of the initial transmission map estimation are improved. Simultaneously, information fusion and skip connections reduce the loss of detailed information, resulting in a more natural and refined dehazing effect. This design has significant advantages in handling complex haze distributions and adapting to various scenario requirements.

[0184] Based on the above embodiments, the pseudocode of the network training method based on spatial transformation and contrastive learning provided in the embodiments of the present invention can be represented as follows:

[0185] enter:

[0186] Input a foggy image

[0187] N: Preset number of iterations

[0188] Output:

[0189] : Initial defogging network parameters

[0190] Initialize the network parameters of the initial defogging network

[0191] for int n=1 to N

[0192] Obtain the first deformed image through the initial spatial transformation network. ;

[0193] Obtain the first clear image through the initial dehazing network. ;

[0194] Obtain the second clear image J through the initial dehazing network;

[0195] Obtain the second deformed image through the initial spatial transformation network. ;

[0196] Obtain the initial atmospheric light map ;

[0197] Obtain the initial transmission graph T;

[0198] Reconstructing the foggy image yields the first reconstruction result. ;

[0199] A clear image is reconstructed, yielding a second reconstruction result. ;

[0200] Calculate training loss ;

[0201] Update network parameters:

[0202] end for

[0203] Return network parameters .

[0204] in, This is the gradient of the training loss.

[0205] like Figure 7 The diagram shown is a complete flowchart of the network training method based on spatial transformation and contrastive learning provided in an embodiment of the present invention. Figure 7 In the process, sample images with fog are input into an initial spatial transformation network, an initial dehazing network, an initial atmospheric illumination map generator, and an initial transmission map generator, respectively, to obtain a first deformed image, a second clear image, an initial atmospheric illumination map, and an initial transmission map. The first deformed image is input into the initial dehazing image to obtain a first clear image. The second clear image is input into the initial spatial transformation network to obtain a second deformed image. The initial atmospheric illumination map and the second clear image are combined with an atmospheric scattering model to obtain a first reconstructed image. Sample images with fog and the initial transmission map are combined with an atmospheric scattering model to obtain a second reconstructed image.

[0206] Using the first clear image and the second deformed image, the processing order consistency constraint can be calculated. Using the sample foggy image, the second clear image, the first reconstruction result, and the second reconstruction result, the bidirectional contrast loss can be calculated.

[0207] Using the second clear image and the sample clear image, a first discrimination result can be obtained through an initial discriminator. Using the second reconstruction result and the sample clear image, a second discrimination result can be obtained through the initial discriminator. Combining the first and second discrimination results, the adversarial loss can be calculated.

[0208] Using the first reconstruction result and the sample foggy image, calculate the content consistency loss; using the second reconstruction result and the second clear image, calculate the clarity consistency loss; using the second reconstruction result and the second clear image, calculate the structural similarity loss.

[0209] In addition, the atmospheric light map estimation loss can be calculated. The training loss can be obtained by weighted summation of the losses calculated above.

[0210] Using this training loss, backpropagation is performed to jointly train the initial discriminator, initial spatial transformation network, initial dehazing network, initial atmospheric light map generator, and initial transport map generator to obtain the final target dehazing network.

[0211] like Figure 8 As shown, based on the above embodiments, this embodiment of the invention provides a network application method based on spatial transformation and contrastive learning, including:

[0212] S21, Obtain the image to be dehazed;

[0213] S22, the image to be dehazed is input into the target dehazing network to obtain the target clear image output by the target dehazing network;

[0214] The target defogging network is trained based on the network training methods based on spatial transformation and contrastive learning provided in the above embodiments.

[0215] Specifically, the network application method based on spatial transformation and contrastive learning provided in this embodiment of the invention is executed by a network application device based on spatial transformation and contrastive learning. This device can be configured within a computer or an image acquisition device. The computer can be a local computer or a cloud computer, and the local computer can be a computer, tablet, etc., without specific limitations. The image acquisition device can be a camera, etc., and can have image dehazing functionality.

[0216] First, step S21 is executed to obtain the image to be dehazed. The image to be dehazed refers to a foggy image that requires dehazing, and the image to be dehazed is a color image.

[0217] Then, step S22 is executed, where the image to be dehazed is input into the target dehazing network, and the target clear image can be obtained and output through the target dehazing network.

[0218] The target dehazing network can be trained using the network training methods based on spatial transformation and contrastive learning provided in the above embodiments.

[0219] The network application method based on spatial transformation and contrastive learning provided in this embodiment of the invention uses a target dehazing network to dehaze the image to be dehazed. Regardless of whether there is geometric perturbation in the image to be dehazed, it can ensure the clarity of the target clear image and achieve the purpose of effectively removing haze from the image to be dehazed.

[0220] like Figure 9 As shown, based on the above embodiments, this embodiment of the invention provides a network training device based on spatial transformation and contrastive learning, comprising:

[0221] The sequential processing module 91 is used to determine, based on the initial spatial transformation network and the initial dehazing network, the first deformed image of the sample hazy image in the sample set and the first clear image of the first deformed image, and based on the initial dehazing network and the initial spatial transformation network, the second clear image of the sample hazy image and the second deformed image of the second clear image, respectively.

[0222] Image decomposition module 92 is used to determine the initial atmospheric light map of the sample foggy image based on the initial atmospheric light map generator, and to determine the initial transmission map of the sample foggy image based on the initial transmission map generator.

[0223] Image reconstruction module 93 is used to reconstruct a foggy image based on the second clear image, the initial atmospheric light map, and the initial transmission map to obtain a first reconstruction result, and to reconstruct a clear image based on the sample foggy image, the initial atmospheric light map, and the initial transmission map to obtain a second reconstruction result;

[0224] The loss calculation module 94 is used to calculate the processing order consistency constraint based on the first clear image and the second deformed image, and to calculate the bidirectional contrast loss based on the sample foggy image, the second clear image, the first reconstruction result and the second reconstruction result;

[0225] The joint training module 95 is used to jointly train the initial spatial transformation network, the initial dehazing network, the initial atmospheric light map generator, and the initial transport map generator based on the processing order consistency constraint and the bidirectional contrast loss, so as to obtain the target dehazing network corresponding to the initial dehazing network.

[0226] Specifically, the functions of each module in the network training device based on spatial transformation and contrastive learning provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0227] like Figure 10 As shown, based on the above embodiments, this embodiment of the invention provides a network application method based on spatial transformation and contrastive learning, including:

[0228] Image acquisition module 101 is used to acquire the image to be dehazed;

[0229] Image dehazing module 102 is used to input the image to be dehazed into the target dehazing network to obtain the target clear image output by the target dehazing network;

[0230] The target defogging network is trained based on the network training methods based on spatial transformation and contrastive learning provided in the above embodiments.

[0231] Specifically, the functions of each module in the network application device based on spatial transformation and contrastive learning provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0232] Figure 11 An example is a schematic diagram of the physical structure of a device, such as... Figure 11As shown, the device may include a processor 110, a communications interface 120, a memory 130, and a communication bus 140, wherein the processor 110, communications interface 120, and memory 130 communicate with each other via the communication bus 140. The processor 110 can call logical instructions in the memory 130 to execute the network training method based on spatial transformation and contrastive learning, or the network application method based on spatial transformation and contrastive learning, provided in the above embodiments.

[0233] Furthermore, the logical instructions in the aforementioned memory 130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0234] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the network training method based on spatial transformation and contrastive learning, or the network application method based on spatial transformation and contrastive learning, provided in the above embodiments.

[0235] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the network training method based on spatial transformation and contrastive learning, or the network application method based on spatial transformation and contrastive learning, provided in the above embodiments.

[0236] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0237] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0238] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network training method based on spatial transformation and contrastive learning, characterized in that, The method comprises the following steps: Based on the initial spatial transformation network and the initial defogging network, the first deformed image of the sample hazy image in the sample set and the first clear image of the first deformed image are determined in turn, and based on the initial defogging network and the initial spatial transformation network, the second clear image of the sample hazy image and the second deformed image of the second clear image are determined in turn; Based on the initial atmospheric light generator, the initial atmospheric light map of the sample hazy image is determined, and based on the initial transmission map generator, the initial transmission map of the sample hazy image is determined; Based on the second clear image, the initial atmospheric light map and the initial transmission map, the hazy image is reconstructed to obtain a first reconstruction result, and based on the sample hazy image, the initial atmospheric light map and the initial transmission map, a clear image is reconstructed to obtain a second reconstruction result; Based on the first clear image and the second deformed image, a processing sequence consistency constraint is calculated, and based on the sample hazy image, the second clear image, the first reconstruction result and the second reconstruction result, a bidirectional contrast loss is calculated; Based on the processing sequence consistency constraint and the bidirectional contrast loss, the initial spatial transformation network, the initial defogging network, the initial atmospheric light map generator and the initial transmission map generator are jointly trained to obtain the target defogging network corresponding to the initial defogging network.

2. The network training method based on spatial transform and contrastive learning according to claim 1, wherein, The method comprises the following steps: Based on the initial discriminator, the second clear image and the second reconstruction result are discriminated respectively to obtain a first discrimination result and a second discrimination result; Based on the first discrimination result and the second discrimination result, an adversarial loss is calculated; Based on the adversarial loss, the processing sequence consistency constraint and the bidirectional contrast loss, the initial spatial transformation network, the initial defogging network, the initial atmospheric light map generator and the initial transmission map generator are jointly trained to obtain the target defogging network.

3. The network training method based on spatial transform and contrastive learning according to claim 1, wherein, The method further comprises the following steps: Based on the first reconstruction result and the sample hazy image, a content consistency loss is calculated, based on the second reconstruction result and the second clear image, a clear consistency loss is calculated, and based on the second reconstruction result and the second clear image, a structural similarity loss is calculated; The initial spatial transformation network, the initial defogging network, the initial atmospheric light map generator and the initial transmission map generator are jointly trained based on the processing sequence consistency constraint and the bidirectional contrast loss, and at least one of the content consistency loss, the clarity consistency loss and the structural similarity loss, to obtain the target defogging network.

4. The network training method based on spatial transform and contrastive learning according to any one of claims 1-3, characterized in that, The initial defogging network comprises a U-Net architecture, which comprises an input convolutional layer, a plurality of down-sampling layers, a plurality of up-sampling layers and an output convolutional layer; A mixed structure block is arranged between the input convolutional layer and the first down-sampling layer, between adjacent two down-sampling layers, between adjacent two up-sampling layers, between the last up-sampling layer and the output convolutional layer, and after the output convolutional layer; the mixed structure block comprises a multi-scale parallel convolution kernel module and a parallel attention module.

5. The network training method based on spatial transform and contrastive learning according to any one of claims 1-3, characterized in that, The initial spatial transformation network comprises a positioning network, a gradient flipping layer, a network generator and a sampler connected in sequence; The positioning network is used to generate a spatial transformation parameter of an input image of the initial spatial transformation network; The gradient flipping layer is used to realize gradient negation when back propagation in the joint training process; The network generator is used to determine a coordinate grid corresponding to an output image of the initial spatial transformation network based on the spatial transformation parameter; the coordinate grid corresponds to each pixel position in the input image of the initial spatial transformation network; The sampler is used to sample the input image of the initial spatial transformation network based on the coordinate grid to obtain the output image of the initial spatial transformation network.

6. The network training method based on spatial transform and contrastive learning according to any one of claims 1-3, characterized in that, The initial atmospheric light map generator comprises an encoder, an intermediate block and a decoder connected in sequence; The encoder is used to extract features of the sample foggy image; The intermediate block is used to convert the features extracted by the encoder into a mean value and a variance of a Gaussian distribution, and sample based on a parameterization technique to obtain a sampling result; The decoder is used to decode the sampling result to obtain the initial atmospheric light map.

7. The network training method based on spatial transform and contrastive learning according to any one of claims 1-3, characterized in that, The initial transmission map generator comprises a first down-sampling layer, a first standard convolution block, a first up-sampling layer, a second down-sampling layer, a first splicing layer, a second standard convolution block, a second up-sampling layer, a third down-sampling layer, a second splicing layer, a third standard convolution block, a third splicing layer and a fourth standard convolution block; The first down-sampling layer, the first standard convolution block and the first up-sampling layer are connected in sequence, the second down-sampling layer, the first splicing layer, the second standard convolution block and the second up-sampling layer are connected in sequence, and the first up-sampling layer is connected with the first splicing layer; The third down-sampling layer, the second splicing layer, the third standard convolution block, the third splicing layer and the fourth standard convolution block are connected in sequence, and the second up-sampling layer is connected with the second splicing layer; The first down-sampling layer, the second down-sampling layer, the third down-sampling layer and the third splicing layer are respectively connected to the sample foggy image, and the fourth standard convolution outputs the initial transmission map.

8. A network application method based on spatial transformation and contrastive learning, characterized in that, The method comprises: acquiring a to-be-defogged image; input the image to be defogged into a target defogging network to obtain a target clear image output by the target defogging network; The target defogging network is trained based on the network training method based on spatial transformation and contrast learning in any one of claims 1-7.

9. An apparatus comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor implements the network training method based on spatial transformation and contrast learning in any one of claims 1-7 or the network application method based on spatial transformation and contrast learning in claim 8 when executing the program. 10.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the network training method based on spatial transformation and contrast learning in any one of claims 1-7 or the network application method based on spatial transformation and contrast learning in claim 8 when executed by the processor.

Images