Zero sample learning defogging image enhancement method and device based on image decomposition
Through the zero-sample learning method of image decomposition, the target layer is generated using the initial network joint training, which solves the problem of difficulty in obtaining image defog model training samples and high time cost, and achieves efficient image defog effect.
Patent Information
- Application Number
- CN202510793853.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-07-11
- Estimated Expiration
- 2045-06-13
AI Technical Summary
The existing image defogging methods have problems such as high training time cost and difficulty in obtaining training samples.
Using the zero-sample learning method based on image decomposition, the joint training of the initial clean layer generation network, the initial atmospheric optical layer generation network, and the initial transmittance layer generation network is reconstructed, and the loss is calculated using the atmospheric scattering model, optimize the network to generate the target layer, and then solve the defogging image.
Without the need for large training samples, training time costs are reduced and image defogging efficiency and accuracy are improved.
Smart Images

Figure CN120298269A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision image enhancement, and particularly to a zero-shot learning dehazing image enhancement method and device based on image decomposition. Background Art
[0002] In recent years, with the decline of air quality and the reduction of visibility, it has had a great impact on people's lives and the development of industry. Under bad weather conditions, such as haze, fog, smoke, etc., the image quality of outdoor scenes often deteriorates. This light mixes with the ambient light reflected in other directions and enters the line of sight through atmospheric particles. Therefore, the contrast of objects photographed under bad weather conditions is low, the color is dull, and the brightness is offset. Image dehazing processing can significantly increase the contrast of objects.
[0003] However, existing image dehazing methods all have problems such as large training time cost of the image dehazing model and difficulty in obtaining training samples. Summary of the Invention
[0004] The present invention provides a zero-shot learning dehazing image enhancement method and device based on image decomposition to solve the defects existing in the prior art.
[0005] The present invention provides a zero-shot learning dehazing image enhancement method based on image decomposition, including: Obtaining a to-be-dehazed image, and respectively inputting the to-be-dehazed image into an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmittance layer generation network to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmittance layer output by the initial transmittance layer generation network; Based on the initial clean layer, the initial atmospheric light layer, and the initial transmittance layer, applying an atmospheric scattering model to reconstruct a haze image, and calculating a reconstruction loss based on the to-be-dehazed image and the reconstructed haze image; Determining the dark channel value of each pixel point in the to-be-dehazed image, based on the dark channel value, determining a true atmospheric light layer corresponding to the to-be-dehazed image, and calculating an atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; Calculating a total loss based on the reconstruction loss and the atmospheric light loss, and jointly optimizing the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network based on the total loss to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmittance layer generation network; Input the image to be dehazed into the target atmospheric light layer generation network and the target transmittance layer generation network to obtain a target atmospheric light layer and a target transmittance layer respectively, and based on the target atmospheric light layer and the target transmittance layer, apply the atmospheric scattering model to solve the dehazed image corresponding to the image to be dehazed.
[0006] According to a zero-shot learning dehazed image enhancement method based on image decomposition provided by the present invention, the initial atmospheric light layer generation network is used to encode the image to be dehazed to obtain a latent variable, determine the normal distribution satisfied by the latent variable, reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable, and decode the reconstructed variable to obtain the initial atmospheric light layer; The method further includes: Calculate the KL divergence loss based on the normal distribution and the standard normal distribution; Calculating the total loss based on the reconstruction loss and the atmospheric light loss specifically includes: Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the KL divergence loss.
[0007] According to a zero-shot learning dehazed image enhancement method based on image decomposition provided by the present invention, the initial atmospheric light layer generation network includes an encoder, an intermediate block, and a decoder, and the encoder and the decoder both include the same number of convolutional blocks; Each convolutional block in the encoder includes a convolutional layer, an activation layer, and a max pooling layer connected in sequence, and the encoder is used to encode the image to be dehazed to obtain a latent variable; The intermediate block is used to determine the normal distribution satisfied by the latent variable, and reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable; Each convolutional block in the decoder includes an upsampling layer, a convolutional layer, a batch normalization layer, and an activation layer connected in sequence, and the decoder is used to decode the reconstructed variable to obtain the initial atmospheric light layer.
[0008] According to a zero-shot learning dehazed image enhancement method based on image decomposition provided by the present invention, calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Calculate the dark channel prior loss based on the L1 norm of the dark channel value; Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the dark channel prior loss.
[0009] According to a zero-shot learning dehazed image enhancement method based on image decomposition provided by the present invention, calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Regularize the initial atmospheric light layer and the initial transmittance layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the regularization loss based on the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the total loss based on the reconstruction loss, the atmospheric light loss and the regularization loss.
[0010] According to a zero-shot learning haze image enhancement method based on image decomposition provided by the present invention, the step of regularizing the initial atmospheric light layer and the initial transmittance layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer includes: Regularize the initial atmospheric light layer and the initial transmittance layer respectively based on the Laplacian norm to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer.
[0011] According to a zero-shot learning haze image enhancement method based on image decomposition provided by the present invention, the initial clean layer generation network is a fully convolutional neural network, and the initial transmittance layer generation network is a U-net architecture.
[0012] The present invention also provides a zero-shot learning haze image enhancement device based on image decomposition, including: A layer generation module, configured to obtain a haze image to be removed, and input the haze image to be removed into an initial clean layer generation network, an initial atmospheric light layer generation network and an initial transmittance layer generation network respectively, to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmittance layer output by the initial transmittance layer generation network; A loss calculation module, configured to apply an atmospheric scattering model based on the initial clean layer, the initial atmospheric light layer and the initial transmittance layer to reconstruct a haze image, and calculate a reconstruction loss based on the haze image to be removed and the reconstructed haze image; The loss calculation module is further configured to determine the dark channel value of each pixel point in the haze image to be removed, determine the true atmospheric light layer corresponding to the haze image to be removed based on the dark channel value, and calculate an atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; A network optimization module, configured to calculate a total loss based on the reconstruction loss and the atmospheric light loss, and jointly optimize the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network based on the total loss to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmittance layer generation network; An image defogging module, configured to input the image to be defogged into the target atmospheric light layer generation network and the target transmittance layer generation network to respectively obtain a target atmospheric light layer and a target transmittance layer, and apply an atmospheric scattering model based on the target atmospheric light layer and the target transmittance layer to solve the defogged image corresponding to the image to be defogged.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method for enhancing a defogged image with zero-shot learning based on image decomposition as described in any one of the above is implemented.
[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for enhancing a defogged image with zero-shot learning based on image decomposition as described in any one of the above is implemented.
[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for enhancing a defogged image with zero-shot learning based on image decomposition as described in any one of the above is implemented.
[0016] Compared with the prior art, the present invention has the following beneficial effects: The method and device for enhancing a defogged image with zero-shot learning based on image decomposition provided by the present invention introduce three initial layer generation networks, namely an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmittance layer generation network. Through joint training, a target atmospheric light layer generation network and a target transmittance layer generation network are obtained, which are respectively used to determine the target atmospheric light layer and the target transmittance layer of the image to be defogged. Then, in combination with the atmospheric scattering model, the defogged image corresponding to the image to be defogged is solved. During the joint training of the three initial layer generation networks, only the image to be defogged is required, and a large number of training samples do not need to be introduced. This can not only avoid the problem of difficult acquisition of training samples for image defogging models in the prior art, but also reduce the amount of calculation during training, greatly shortening the training duration of the network and reducing the training time cost of the network. Description of the Drawings
[0017] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0018] Figure 1 is one of the flow schematic diagrams of the zero-shot learning haze-removing image enhancement method based on image decomposition provided by the present invention; Figure 2 is the flow schematic diagram of generating the target clean layer, the target atmospheric light layer and the target transmittance layer in the zero-shot learning haze-removing image enhancement method based on image decomposition provided by the present invention; Figure 3 is the structural schematic diagram of the initial A-Net in the zero-shot learning haze-removing image enhancement method based on image decomposition provided by the present invention; Figure 4 is the structural schematic diagram of the initial Clear-Net in the zero-shot learning haze-removing image enhancement method based on image decomposition provided by the present invention; Figure 5 is the structural schematic diagram of the initial T-Net in the zero-shot learning haze-removing image enhancement method based on image decomposition provided by the present invention; Figure 6 is the structural schematic diagram of the zero-shot learning haze-removing image enhancement device provided by the present invention; Figure 7 is the structural schematic diagram of the electronic device provided by the present invention. Specific Embodiments
[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0020] Figure 1 is the flow schematic diagram of a zero-shot learning haze-removing image enhancement method provided in an embodiment of the present invention. As Figure 1 shown, the method includes: S1. Obtain the image to be dehazed, and input the image to be dehazed into an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmittance layer generation network respectively, to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmittance layer output by the initial transmittance layer generation network; S2. Based on the initial clean layer, the initial atmospheric light layer, and the initial transmittance layer, apply an atmospheric scattering model to reconstruct a hazy image, and calculate a reconstruction loss based on the image to be dehazed and the reconstructed hazy image; S3. Determine the dark channel value of each pixel point in the image to be dehazed, based on the dark channel value, determine the true atmospheric light layer corresponding to the image to be dehazed, and calculate an atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; S4. Calculate a total loss based on the reconstruction loss and the atmospheric light loss, and based on the total loss, jointly optimize the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network, to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmittance layer generation network; S5. Input the image to be dehazed into the target atmospheric light layer generation network and the target transmittance layer generation network, respectively obtain a target atmospheric light layer and a target transmittance layer, and based on the target atmospheric light layer and the target transmittance layer, apply an atmospheric scattering model to solve the dehazed image corresponding to the image to be dehazed.
[0021] Specifically, in the embodiments of the present invention, the zero-shot learning dehazed image enhancement method based on image decomposition, the execution subject of which is a zero-shot learning dehazed image enhancement device based on image decomposition. This device can be configured in an electronic device, which can be a computer or an image acquisition device. The computer can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., and no specific limitation is made here. The image acquisition device can be a camera, which can have the zero-shot learning dehazed image enhancement function based on image decomposition.
[0022] It can be understood that the zero-shot learning dehazed image enhancement function refers to realizing image enhancement by the way of zero-shot learning the image dehazing process, to obtain the dehazed image corresponding to the image to be dehazed. This dehazed image is the result of image enhancement.
[0023] First, execute step S1 to obtain the image to be dehazed. The image to be dehazed refers to a foggy image with a dehazing requirement, and this image to be dehazed is a color image.
[0024] The image to be dehazed can be input into the initial clear layer generation network (Clear-Net), the initial atmospheric light layer generation network (A-Net), and the initial transmittance layer generation network (T-Net) respectively, to obtain the initial clear layer output by the initial clear layer generation network, the initial atmospheric light layer output by the initial atmospheric light layer generation network, and the initial transmittance layer output by the initial transmittance layer generation network. Thus, through the initial Clear-Net, the initial A-Net, and the initial T-Net, the image to be dehazed can be disentangled into the initial clear layer, the initial atmospheric light layer, and the initial transmittance layer.
[0025] After that, step S2 is executed. Using the initial clear layer, the initial atmospheric light layer, and the initial transmittance layer, the hazy image is reconstructed by applying the atmospheric scattering model.
[0026] The atmospheric scattering model can be expressed by the following formula: ; where, is the hazy image, is the clear layer of the hazy image, is the transmittance layer of the hazy image, is the atmospheric light layer of the hazy image, and a is the pixel point.
[0027] Using the image x to be dehazed and the hazy image f, the reconstruction loss is calculated. That is:
[0028] where, n is the number of pixel points in the image x to be dehazed and the hazy image f, is the pixel value of the pixel point i in the hazy image f, is the pixel value of the pixel point i in the image x to be dehazed, is the reconstruction loss.
[0029] The purpose of this reconstruction loss is to decompose the image to be dehazed into three different image layers, namely the clear layer, the atmospheric light layer, and the transmittance layer, by minimizing the difference between the reconstructed hazy image and the image to be dehazed.
[0030] After that, step S3 is executed to determine the dark channel value of each pixel point in the image to be dehazed. That is: ; where, is the dark channel value of the pixel point i in the image to be dehazed, is the local image patch centered on the pixel point i in the image to be dehazed, that is, a small patch around the pixel point i, c is the color channel, r is the red channel, g is the green channel, and b is the blue channel, is the pixel value of the pixel point y in the color channel c of the image to be dehazed, where y is the pixel point in
[0031] Using the dark channel value, the true atmospheric light layer corresponding to the image to be dehazed can be determined. There is: ; where is the true atmospheric light layer corresponding to the image to be dehazed, is a set of a preset number of pixel points with high dark channel values, is the pixel value of the pixel point j in the image to be dehazed.
[0032] Using the true atmospheric light layer and the initial atmospheric light layer , calculate the atmospheric light loss. There is: ; where is the atmospheric light loss, which belongs to the mean square error loss.
[0033] Finally, execute step S4. Using the reconstruction loss and the atmospheric light loss, calculate the total loss. Here, the reconstruction loss and the atmospheric light loss can be weighted and summed to obtain the total loss, or other losses can be introduced according to needs based on the reconstruction loss and the atmospheric light loss, and the weighted sum of all losses can be calculated to obtain the total loss.
[0034] Using the total loss, the network for generating the initial clean layer, the network for generating the initial atmospheric light layer, and the network for generating the initial transmittance layer can be jointly optimized to obtain the network for generating the target clean layer, the network for generating the target atmospheric light layer, and the network for generating the target transmittance layer.
[0035] The network for generating the target clean layer has the performance of generating the target clean layer of the image to be dehazed, the network for generating the target atmospheric light layer has the performance of generating the target atmospheric light layer of the image to be dehazed, and the network for generating the target transmittance layer has the performance of generating the target transmittance layer of the image to be dehazed.
[0036] Finally, execute step S5. Input the image to be dehazed into the network for generating the target atmospheric light layer and the network for generating the target transmittance layer to obtain the target atmospheric light layer and the target transmittance layer respectively, and use the target atmospheric light layer and the target transmittance layer to apply the atmospheric scattering model to solve the dehazed image corresponding to the image to be dehazed. As Figure 2 shown.
[0037] It can be understood that although the target clean layer generation network has the performance of generating the target clean layer of the image to be dehazed, in the embodiments of the present invention, the dehazed image corresponding to the image to be dehazed is not directly determined by using the target clean layer generation network. The target clean layer generation network is only an intermediate product obtained by jointly optimizing the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network.
[0038] In the method for enhancing dehazed images with zero-shot learning based on image decomposition provided in the embodiments of the present invention, three initial layer generation networks, namely, the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network, are introduced. Through joint training, the target atmospheric light layer generation network and the target transmittance layer generation network are obtained, which are respectively used to determine the target atmospheric light layer and the target transmittance layer of the image to be dehazed. Then, in combination with the atmospheric scattering model, the dehazed image corresponding to the image to be dehazed is solved. During the joint training of the three initial layer generation networks, only the image to be dehazed is required, and a large number of training samples do not need to be introduced. This can not only avoid the problem of difficult acquisition of training samples in the existing image dehazing model, but also reduce the computational complexity during training, greatly shortening the training duration of the network and reducing the training time cost of the network.
[0039] In the above embodiment, the initial atmospheric light layer generation network is used to encode the image to be dehazed to obtain a latent variable, determine the normal distribution satisfied by the latent variable, reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable, and decode the reconstructed variable to obtain the initial atmospheric light layer; The method further includes: Calculating the KL divergence loss based on the normal distribution and the standard normal distribution; The calculating of the total loss based on the reconstruction loss and the atmospheric light loss specifically includes: Calculating the total loss based on the reconstruction loss, the atmospheric light loss, and the KL divergence loss.
[0040] Specifically, after the image to be dehazed is input into the initial atmospheric light layer generation network, the initial atmospheric light layer generation network can encode the image to be dehazed to obtain a latent variable, and can convert the latent variable into the mean and variance of the normal distribution by minimizing an equation, determine the normal distribution satisfied by the latent variable. Then, through reparameterization of the normal distribution, the latent variable is reconstructed by resampling in the normal distribution to obtain a reconstructed variable, and the reconstructed variable is decoded to obtain the initial atmospheric light layer.
[0041] Here, in order to perform end-to-end optimization using the standard stochastic gradient method, the variational lower bound is reparameterized so that a lower bound estimator that can be directly optimized using the standard stochastic gradient method can be obtained.
[0042] Furthermore, the method can also introduce the KL divergence loss, which can be calculated from the normal distribution and the standard normal distribution, i.e., there is: ; where is the KL (Kullback-Leibler) divergence loss, used to force the latent variable to conform to the standard normal distribution , is the normal distribution 's mean, is the normal distribution 's variance. KL represents the KL divergence, used to measure the difference between two probability distributions, and || represents the calculation of the KL divergence between two probability distributions. represents the i-th dimension of the latent variable z.
[0043] After that, the total loss can be calculated by weighted summing the reconstruction loss, the atmospheric light loss, and the KL divergence loss.
[0044] In the embodiments of the present invention, including the KL divergence loss in the total loss can improve the performance of the target atmospheric light layer generation network in generating the target atmospheric light layer.
[0045] Based on the above embodiments, the initial atmospheric light layer generation network includes an encoder, an intermediate block, and a decoder, and the encoder and the decoder include the same number of blocks; Each block in the encoder includes a convolutional layer, an activation layer, and a max pooling layer connected in sequence, and the encoder is used to encode the defogging image to obtain a latent variable; The intermediate block is used to determine the normal distribution satisfied by the latent variable, and reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable; Each block in the decoder includes an upsampling layer, a convolutional layer, a batch normalization layer, and an activation layer connected in sequence, and the decoder is used to decode the reconstructed variable to obtain the initial atmospheric light layer.
[0046] Specifically, the initial A-Net can be a variational autoencoder structure, including an encoder, an intermediate block, and a decoder. The encoder can be constructed based on a Convolutional Neural Networks (CNN), and the decoder can be symmetric with the encoder. Both the encoder and the decoder include the same number of convolutional blocks, for exampleFigure 3 As shown, both the encoder and the decoder include 4 convolutional blocks.
[0047] Each convolutional block in the encoder includes a 3×3 convolutional layer (conv), a ReLU activation layer, and a MaxPooling layer connected in sequence. The encoder is used to encode the dehazing image to obtain a latent variable.
[0048] The middle block is used to determine the normal distribution satisfied by the latent variable, and reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable.
[0049] Each convolutional block in the decoder includes an UpSample layer, a 3×3 convolutional layer, a BatchNorm layer, and a ReLU activation layer connected in sequence. The decoder is used to decode the reconstructed variable to obtain an initial atmospheric light layer.
[0050] In addition, the initial A-Net may further include an output layer, which may include an UpSample layer, a 3×3 convolutional layer, and a Sigmoid activation layer connected in sequence, for outputting the initial atmospheric light layer.
[0051] Based on the above embodiments, calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Calculating a dark channel prior loss based on the L1 norm of the dark channel value; Calculating the total loss based on the reconstruction loss, the atmospheric light loss, and the dark channel prior loss.
[0052] Specifically, the method may further introduce a dark channel prior loss to constrain the Clear-Net network to form a clean layer with dark channel statistical characteristics.
[0053] The dark channel prior loss can be calculated from the dark channel value, that is: ; where is the dark channel prior loss, is the L1 norm.
[0054] Thereafter, the total loss can be calculated by weighted summing the reconstruction loss, the atmospheric light loss, and the dark channel prior loss. It can be understood that in the presence of KL divergence loss, the reconstruction loss, the atmospheric light loss, the KL divergence loss, and the dark channel prior loss can be weighted summed to calculate the total loss, improving the comprehensive performance of the three target layer generation networks.
[0055] Based on the above embodiments, calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Regularize the initial atmospheric light layer and the initial transmittance layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the regularization loss based on the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the regularization loss.
[0056] Specifically, the method can also introduce a regularization loss to improve the stability of the three target layer generation networks.
[0057] To obtain the regularization loss, the initial atmospheric light layer and the initial transmittance layer can be regularized respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer, and then the regularized result of the atmospheric light layer and the regularized result of the transmittance layer are weighted and summed to obtain the regularization loss. There is: ; Among them, is the regularization loss, is the initial atmospheric light layer, is the initial transmittance layer, is the regularized result of the atmospheric light layer, is the regularized result of the transmittance layer, 、 are balance factors, used as weights for weighting, and there is , .
[0058] After that, the reconstruction loss, the atmospheric light loss, and the regularization loss are weighted and summed to calculate the total loss. It can be understood that in the case of the KL divergence loss, the reconstruction loss, the atmospheric light loss, the regularization loss, and the KL divergence loss can be weighted and summed to obtain the total loss. In the case of the KL divergence loss and the dark channel prior loss, the reconstruction loss, the atmospheric light loss, the regularization loss, the KL divergence loss, and the dark channel prior loss can be weighted and summed to calculate the total loss, further improving the comprehensive performance of the three target layer generation networks. At this time, the total loss can be expressed as: .
[0059] Among them, is the total loss, is the atmospheric light loss.
[0060] Based on the above embodiments, the regularization of the initial atmospheric light layer and the initial transmittance layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer includes: Based on the Laplacian norm, the initial atmospheric light layer and the initial transmittance layer are regularized respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer.
[0061] Specifically, the regularized result of the atmospheric light layer can be expressed as:
[0062] The regularized result of the transmittance layer can be expressed as:
[0063] Wherein, is the second-order neighborhood of, is the second-order neighborhood of, and both represent the neighborhood size.
[0064] Using the Laplacian norm, the regularization process plays the role of mean filtering, making the obtained target atmospheric light layer and target transmittance layer smooth.
[0065] Based on the above embodiments, the initial clean layer generation network is a fully convolutional neural network, and the initial transmittance layer generation network is a U-net architecture.
[0066] Specifically, as Figure 4 shown, the initial Clear-Net can be a fully convolutional network, including a plurality of convolutional activation blocks connected in sequence, a 3×3 convolutional layer, and an activation layer (Sigmoid). Each convolutional activation layer includes a 3×3 convolutional layer and an activation layer (Relu). As Figure 5 shown, the initial T-Net can be a U-net architecture, including a compression path, an expansion path, and a 1×1 convolutional layer. The compression path includes 5 first-type convolutional blocks of 3×3 connected in sequence. The expansion path includes 5 second-type convolutional blocks of 3×3 connected in sequence. The first-type convolutional block includes a padding layer, a 3×3 convolutional layer, a batch normalization layer, an activation layer (Leaky ReLU), a padding layer, a 3×3 convolutional layer, a batch normalization layer, and an activation layer (Leaky ReLU) connected in sequence.
[0067] The second type of convolutional block includes an upsampling layer, a batch normalization layer, a padding layer, a 3×3 convolutional layer, a batch normalization layer, an activation layer (Leaky ReLU), a padding layer, a 1×1 convolutional layer, a batch normalization layer, and an activation layer (Leaky ReLU) connected in sequence.
[0068] In summary, in the embodiment of the present invention, the zero-shot learning haze image enhancement method based on image decomposition uses a three-branch network to decompose the input hazy image into three parts, namely, a clean image layer, an atmospheric light layer, and a transmittance layer. At the same time, the three-branch network is learned and decomposed only according to one input hazy image to be dehazed, avoiding the problems of difficult sample collection and high network training time cost, which is very important for subsequent image processing steps.
[0069] The image is decomposed through the network, and the output of the initial Clear-Net is constrained to have statistical characteristics using the dark channel prior loss. The output of the initial A-Net is constrained using the KL divergence loss and the dark channel prior loss. The output of the initial T-Net is constrained through the regularization loss. The three target layers are used to synthesize a hazy image through the atmospheric scattering model, and the reconstruction loss between the synthesized hazy image and the input hazy image to be dehazed is used to guide the learning to generate accurate target atmospheric light layer and target transmittance layer. Finally, the dehazed image is obtained according to the target atmospheric light layer and the target transmittance layer. The three initial networks adopted by this method do not need to be trained with a large amount of data sets. They can directly keep the content details of the clear image as much as possible from various interferences according to one input hazy image to be dehazed, and estimate the final dehazed image, which provides efficiency and robustness for subsequent work, avoids intensive data collection and the problem of domain transfer in the real world using synthetic foggy images, and provides more reliable data and a higher level of automation for industrial production and manufacturing.
[0070] As Figure 6 shown, on the basis of the above embodiment, an apparatus for enhancing a zero-shot learning haze image based on image decomposition is provided in an embodiment of the present invention, including: A layer generation module 61, configured to obtain a hazy image to be dehazed, and input the hazy image to be dehazed into an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmittance layer generation network respectively, to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmittance layer output by the initial transmittance layer generation network; A loss calculation module 62, configured to apply an atmospheric scattering model based on the initial clean layer, the initial atmospheric light layer, and the initial transmittance layer to reconstruct a hazy image, and calculate a reconstruction loss based on the hazy image to be dehazed and the reconstructed hazy image; The loss calculation module 62 is further configured to determine the dark channel values of each pixel point in the to-be-dehazed image, determine the true atmospheric light layer corresponding to the to-be-dehazed image based on the dark channel values, and calculate the atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; The network optimization module 63 is configured to calculate the total loss based on the reconstruction loss and the atmospheric light loss, and jointly optimize the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network based on the total loss to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmittance layer generation network; The image dehazing module 64 is configured to input the to-be-dehazed image into the target atmospheric light layer generation network and the target transmittance layer generation network to respectively obtain a target atmospheric light layer and a target transmittance layer, and apply an atmospheric scattering model based on the target atmospheric light layer and the target transmittance layer to solve the dehazed image corresponding to the to-be-dehazed image.
[0071] Based on the above embodiments, the initial atmospheric light layer generation network is configured to encode the to-be-dehazed image to obtain a latent variable, determine the normal distribution satisfied by the latent variable, reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable, and decode the reconstructed variable to obtain the initial atmospheric light layer; The network optimization module is further configured to: Calculate the KL divergence loss based on the normal distribution and the standard normal distribution; The calculating the total loss based on the reconstruction loss and the atmospheric light loss specifically includes: Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the KL divergence loss.
[0072] Based on the above embodiments, the initial atmospheric light layer generation network includes an encoder, an intermediate block, and a decoder, and the encoder and the decoder include the same number of convolutional blocks; Each convolutional block in the encoder includes a convolutional layer, an activation layer, and a max pooling layer connected in sequence, and the encoder is configured to encode the to-be-dehazed image to obtain a latent variable; The intermediate block is configured to determine the normal distribution satisfied by the latent variable, reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable; Each convolutional block in the decoder includes an upsampling layer, a convolutional layer, a batch normalization layer, and an activation layer connected in sequence, and the decoder is configured to decode the reconstructed variable to obtain the initial atmospheric light layer.
[0073] Based on the above embodiments, the network optimization module is further configured to: Calculate the dark channel prior loss based on the L1 norm of the dark channel value; Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the dark channel prior loss.
[0074] Based on the above embodiments, the network optimization module is further configured to: Regularize the initial atmospheric light layer and the initial transmittance layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the regularization loss based on the regularized result of the atmospheric light layer and the regularized result of the transmittance layer; Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the regularization loss.
[0075] Based on the above embodiments, the network optimization module is further configured to: Regularize the initial atmospheric light layer and the initial transmittance layer respectively based on the Laplacian norm to obtain the regularized result of the atmospheric light layer and the regularized result of the transmittance layer.
[0076] Based on the above embodiments, the initial clean layer generation network is a fully convolutional neural network, and the initial transmittance layer generation network is a U-net architecture.
[0077] Specifically, in the zero-shot learning dehazing image enhancement device based on image decomposition provided in the embodiments of the present invention, the functions of each module correspond one-to-one to the operation processes of each step in the above method embodiments, and the achieved effects are also the same. For details, refer to the above embodiments, and the embodiments of the present invention will not be elaborated herein.
[0078] Figure 7 An example of the physical structure diagram of an electronic device is shown as Figure 7 As shown, the electronic device may include: a processor (Processor) 710, a communication interface (Communications Interface) 720, a memory (Memory) 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the zero-shot learning dehazing image enhancement method based on image decomposition provided in each of the above embodiments.
[0079] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.
[0080] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the zero-shot learning haze removal image enhancement method based on image decomposition provided in the above-mentioned various embodiments.
[0081] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the zero-shot learning haze removal image enhancement method based on image decomposition provided in the above-mentioned various embodiments.
[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.
[0083] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disks, optical discs, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.
[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A zero - sample learning de - fogging image enhancement method based on image decomposition, characterized in that, Including: Obtain the image to be dehazed, and input the image to be dehazed into an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmittance layer generation network respectively, to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmittance layer output by the initial transmittance layer generation network; Based on the initial clean layer, the initial atmospheric light layer, and the initial transmittance layer, apply the atmospheric scattering model to reconstruct the hazy image, and calculate the reconstruction loss based on the image to be dehazed and the reconstructed hazy image; Determine the dark channel value of each pixel point in the image to be dehazed, based on the dark channel value, determine the true atmospheric light layer corresponding to the image to be dehazed, and calculate the atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; Calculate the total loss based on the reconstruction loss and the atmospheric light loss, and jointly optimize the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmittance layer generation network based on the total loss to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmittance layer generation network; Input the image to be dehazed into the target atmospheric light layer generation network and the target transmittance layer generation network to obtain a target atmospheric light layer and a target transmittance layer respectively, and apply the atmospheric scattering model based on the target atmospheric light layer and the target transmittance layer to solve the dehazed image corresponding to the image to be dehazed.
2. The zero-sample learning haze image enhancement method based on image decomposition according to claim 1, wherein, The initial atmospheric light layer generation network is used to encode the image to be dehazed to obtain a latent variable, determine the normal distribution satisfied by the latent variable, reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable, and decode the reconstructed variable to obtain the initial atmospheric light layer; The method further includes: Calculate the KL divergence loss based on the normal distribution and the standard normal distribution; The calculating the total loss based on the reconstruction loss and the atmospheric light loss specifically includes: Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the KL divergence loss.
3. The zero-sample learning haze image enhancement method based on image decomposition according to claim 2, wherein, The initial atmospheric light layer generation network includes an encoder, an intermediate block, and a decoder, and the encoder and the decoder both include the same number of convolutional blocks; Each convolutional block in the encoder includes a convolutional layer, an activation layer, and a max pooling layer connected in sequence, and the encoder is used to encode the image to be dehazed to obtain a latent variable; The intermediate block is used to determine the normal distribution satisfied by the latent variable, and reconstruct the latent variable based on the normal distribution to obtain a reconstructed variable; Each convolutional block in the decoder includes an upsampling layer, a convolutional layer, a batch normalization layer, and an activation layer connected in sequence, and the decoder is used to decode the reconstructed variable to obtain the initial atmospheric light layer.
4. The zero-shot learning haze image enhancement method based on image decomposition according to claim 1, wherein The calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Calculate the dark channel prior loss based on the L1 norm of the dark channel value. Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the dark channel prior loss.
5. The zero-sample learning haze image enhancement method based on image decomposition according to claim 1, characterized in that Calculating the total loss based on the reconstruction loss and the atmospheric light loss further includes: Regularize the initial atmospheric light layer and the initial transmission rate layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmission rate layer; Calculate the regularization loss based on the regularized result of the atmospheric light layer and the regularized result of the transmission rate layer; Calculate the total loss based on the reconstruction loss, the atmospheric light loss, and the regularization loss.
6. The zero-shot learning dehazing image enhancement method based on image decomposition according to claim 5, wherein The step of regularizing the initial atmospheric light layer and the initial transmission rate layer respectively to obtain the regularized result of the atmospheric light layer and the regularized result of the transmission rate layer includes: Regularize the initial atmospheric light layer and the initial transmission rate layer respectively based on the Laplacian norm to obtain the regularized result of the atmospheric light layer and the regularized result of the transmission rate layer.
7. The zero-shot learning haze image enhancement method based on image decomposition according to any one of claims 1-6, characterized in that The initial clean layer generation network is a fully convolutional neural network, and the initial transmission rate layer generation network is a U-net architecture.
8. A zero-shot learning dehazing image enhancement device based on image decomposition, characterized in that, It includes: A layer generation module for obtaining a foggy image to be dehazed and inputting the foggy image to an initial clean layer generation network, an initial atmospheric light layer generation network, and an initial transmission rate layer generation network respectively to obtain an initial clean layer output by the initial clean layer generation network, an initial atmospheric light layer output by the initial atmospheric light layer generation network, and an initial transmission rate layer output by the initial transmission rate layer generation network; A loss calculation module for reconstructing a hazy image based on the initial clean layer, the initial atmospheric light layer, and the initial transmission rate layer by applying an atmospheric scattering model, and calculating a reconstruction loss based on the foggy image to be dehazed and the reconstructed hazy image; The loss calculation module is further configured to determine the dark channel value of each pixel point in the foggy image to be dehazed, determine the true atmospheric light layer corresponding to the foggy image to be dehazed based on the dark channel value, and calculate the atmospheric light loss based on the true atmospheric light layer and the initial atmospheric light layer; A network optimization module for calculating the total loss based on the reconstruction loss and the atmospheric light loss, and jointly optimizing the initial clean layer generation network, the initial atmospheric light layer generation network, and the initial transmission rate layer generation network based on the total loss to obtain a target clean layer generation network, a target atmospheric light layer generation network, and a target transmission rate layer generation network; An image dehazing module for inputting the foggy image to be dehazed into the target atmospheric light layer generation network and the target transmission rate layer generation network to obtain a target atmospheric light layer and a target transmission rate layer respectively, and solving the dehazed image corresponding to the foggy image to be dehazed based on the target atmospheric light layer and the target transmission rate layer by applying an atmospheric scattering model.
9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the zero-shot learning dehazing image enhancement method based on image decomposition according to any one of claims 1-7.
10. A non-transitory computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for enhancing a defogged image in zero-shot learning based on image decomposition according to any one of claims 1-7.
Citation Information
Patent Citations
Traffic image defogging method based on improved generative adversarial network
CN112801902A
Unsupervised image defogging enhancement method and system based on decoupling
CN116542867A
Two-stage decoupling image defogging method based on zero sample learning
CN116757957A
Multi-scale image defogging method suitable for camera device
CN119599911A
Generative adversarial networks for transformer-based dehazing
US20250191146A1
Cited By
Network training and application method and equipment based on spatial transformation and comparative learning
CN120852243A
Glandular cystitis operation imaging interference processing method
CN121095114A
Zero-reference retina image enhancement method, system and device and medium
CN121120441A
A zero-reference retinal image enhancement method, system, device, and medium
CN121120441B
Video restoration method and device, electronic equipment and storage medium
CN121563818A
