A deep learning image defogging method based on single image self-supervision

Abstract

The application relates to the technical field of image processing, and discloses a deep learning image defogging method based on single-image self-supervision, which comprises the following steps: obtaining a to-be-processed fog image, and performing self-supervision pretreatment on the to-be-processed fog image to obtain a self-supervision fog image; a loss function of the network model comprises two parts of defogging constraints and denoising constraints; the self-supervision fog image is used to train a self-supervision network model designed by the application; after multiple rounds of iterative training, a defogging model and a defogged image after reconstruction are obtained. The loss function is jointly designed based on the defogging constraints and the denoising constraints, the defogging constraints realize image defogging mapping, the denoising constraints eliminate noise, the self-supervision training based on a single fog image realizes high-quality image defogging reconstruction, and the application has the characteristics of high image recovery quality, strong generalization ability and strong practicability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a deep learning-based image dehazing method based on self-supervised single-image processing. Background Technology

[0002] In adverse weather conditions, the scattering effect of particles such as dust, haze, and water droplets alters the light propagation path, leading to reduced visibility and severely degrading the quality of outdoor images. This manifests as image blurring and contrast loss, consequently affecting the performance of various computer vision tasks, such as object detection, tracking, and environmental monitoring. Therefore, image dehazing technology has become a highly challenging and practical research topic in the field of computer vision. Dehazing a single image is considered a problem of solving ill-conditioned equations because the relationship between a foggy image and its corresponding clear image involves multiple unknown variables, such as the atmospheric scattering coefficient and lighting conditions.

[0003] To address this problem, researchers have proposed various image dehazing and restoration algorithms, mainly falling into three categories: image enhancement algorithms, image restoration algorithms, and deep learning-based image dehazing algorithms. Among these, image enhancement algorithms are represented by histogram equalization and Retinex algorithms. These algorithms are computationally efficient but tend to amplify noise, resulting in limited dehazing effects. Image restoration algorithms are represented by Dark Channel Prior (DCP) algorithms, which are based on atmospheric scattering models and estimate atmospheric transmittance using dark channel priors from naturally hazy images. However, the dark channel assumption in DCP algorithms leads to their failure when sky regions are present in the image. Deep learning-based image dehazing algorithms provide data-driven solutions, effectively addressing the dependence of traditional methods on physical model solutions. A typical example is supervised deep learning models based on convolutional neural networks, whose core advantage lies in their ability to learn the complex mapping relationship between hazy and clear scenes from large-scale data.

[0004] The methods described above require a large number of paired fog images and clear images for supervised training, but such paired data is extremely difficult to obtain in real-world scenarios. Most existing studies use synthetic fog images as an alternative, but their physical degradation patterns differ significantly from real fog and haze, leading to domain shift issues that limit the network's generalization ability and result in poor practicality. Summary of the Invention

[0005] This invention provides a deep learning-based image dehazing method based on self-supervised single image, which solves the problem of poor practicality of existing dehazing models that use synthetic fog images to replace eukaryotic haze images.

[0006] In a first aspect, the present invention provides a deep learning-based image dehazing method based on self-supervised single-image processing, the method comprising: Obtain the fog image to be processed, and perform self-supervised preprocessing on the fog image to obtain a self-supervised fog image; A loss function is designed based on a combination of defogging and denoising constraints, and a self-supervised network model is constructed. By using self-supervised fog maps and iteratively training a self-supervised network model based on a loss function, a dehazing model and a reconstructed dehazing image are obtained. The present invention provides a deep learning image dehazing method based on self-supervised training of a single image. The loss function is designed based on the joint design of dehazing constraints and denoising constraints. The dehazing constraints realize the image dehazing mapping, and the denoising constraints eliminate noise. High-quality image dehazing reconstruction is achieved based on self-supervised training of a single fog map. It overcomes the shortcomings of traditional deep learning, such as dependence on large-scale datasets and poor generalization ability. It has the characteristics of high image restoration quality, strong generalization ability and strong practicality.

[0007] In one optional implementation, a self-supervised preprocessing is performed on the fog map to be processed to obtain a self-supervised fog map, including: Diagonal resampling is performed on the fog image to be processed to obtain two mutually supervised images; The mutual supervision image is upsampled using the Fourier domain zero-filling method so that the size of the mutual supervision image is the same as that of the fog image to be processed; Data augmentation is performed based on mutually supervised images to obtain a self-supervised fog map, which includes the image sets corresponding to the two mutually supervised images.

[0008] In one optional implementation, the fog image to be processed is resampled diagonally to obtain two mutually supervised images, including: Based on the preset division principle, multiple adjacent pixels in the fog image to be processed are divided into a square unit, and multiple square units are obtained by traversing each pixel in the fog image to be processed. Calculate the average pixel value on the diagonal of each square unit, and use it as the first new pixel and the second new pixel respectively; Two mutually supervised images are constructed using the first new pixel and the second new pixel, respectively.

[0009] The deep learning image dehazing method based on self-supervised single image provided by this invention obtains two mutually supervised images through diagonal resampling, matches the size of the original fog map using Fourier domain zero-padding upsampling, and then generates a self-supervised fog map through data augmentation. This generates image data that meets the requirements of self-supervised training and has suitable detail and noise characteristics, providing high-quality samples for subsequent dehazing model training.

[0010] In one optional implementation, the self-supervised fog map includes a first self-supervised image and a second self-supervised image, wherein the first self-supervised image and the second supervised image have the same image content and the noise is distributed in opposite directions. The dehazing process based on dehazing constraints includes: The second self-supervised image is dehazed using the Retinex algorithm to obtain the second dehazed image. Substitute the second dehazed image into the visual perception model to obtain the second hazy image; The dehazing constraint part of the loss function is constructed with the objective of minimizing the distance between the second foggy image and the first self-supervised image, and dehazing is achieved based on the dehazing constraint part of the self-supervised network model.

[0011] In one optional implementation, the process of partial line-of-sight denoising based on denoising constraints includes: The first self-supervised image and the second self-supervised image are respectively input into the self-supervised network model to obtain the first network output image and the second network output image; Assuming the noise median of the first network output image and the second output image is 0, the denoising constraint part of the loss function is constructed with the goal of minimizing the distance between the first network output image and the second self-supervised image. Denoising is achieved by using the variational method and the denoising constraint part of the self-supervised network model.

[0012] The present invention provides a deep learning-based image dehazing method based on self-supervised single images. The loss function is designed by combining dehazing and visual perception model operations. It incorporates dehazing constraints to ensure the direction of dehazing, and also utilizes denoising constraints. Based on image segmentation, distance minimization, variational methods, and noise independence, it achieves denoising under the assumption that the noise median is 0. This allows the self-supervised network model to simultaneously consider the accuracy of dehazing and the effectiveness of denoising, improving the model's processing quality of foggy images and enhancing the clarity and purity of the dehazed image.

[0013] In one alternative implementation, the self-supervised network model is a convolutional neural network architecture. Constructing the self-supervised network model includes: Convolutional neural network architecture is constructed, which includes: encoder module, decoder module, and hopping module.

[0014] The encoder module is connected to the nonlinear activation unit and the max pooling unit in sequence, and the decoder module is connected to the nonlinear activation unit and the bilinear interpolation unit in sequence.

[0015] In one optional implementation, a dehazing model is obtained by training a self-supervised network model based on a loss function using a self-supervised fog map, including: The self-supervised fog map is input into the self-supervised network model for iterative training, and the corresponding loss function is calculated each time. The training continues until the loss function converges or meets the preset conditions, resulting in a well-trained dehazing model.

[0016] The present invention provides a deep learning-based image dehazing method based on self-supervised single image. It uses a convolutional neural network architecture to construct a self-supervised network model. The encoder module combines nonlinear activation and max pooling, while the decoder module combines nonlinear activation and bilinear interpolation. This effectively extracts fog map features and accurately restores image details. Using the self-supervised fog map, the model is iteratively trained based on the loss function until the loss function converges. This allows the trained dehazing model to accurately learn the dehazing rules, improve the dehazing effect on fog maps, and make the dehazed image clearer.

[0017] Secondly, the present invention provides a deep learning image dehazing device based on self-supervised single-image processing, the device comprising: The image preprocessing module is used to acquire the fog image to be processed and perform self-supervised preprocessing on the fog image to obtain a self-supervised fog image. The loss function design module is used to design loss functions based on dehazing and denoising constraints, and to build a self-supervised network model. The model training and dehazing module is used to iteratively train a self-supervised network model using a self-supervised fog map to obtain a dehazed model and a reconstructed dehazed image.

[0018] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first process of a deep learning image dehazing method based on self-supervised single image according to an embodiment of the present invention; Figure 3This is a schematic diagram of the second process of a deep learning image dehazing method based on self-supervised single image according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the resampling of the original hazy image in a deep learning image dehazing method based on single-image self-supervision according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating a specific embodiment of a deep learning image dehazing method based on self-supervised single-image processing according to an embodiment of the present invention. Figure 6 This is a structural block diagram of a deep learning image dehazing device based on self-supervised single-image processing according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0024] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0025] As an optional application scenario of this invention, such as Figure 1 As shown, this deep learning image dehazing system based on single-image self-supervision may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0026] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0027] This invention provides a deep learning-based image dehazing method based on self-supervised single-image processing. By jointly designing a loss function using dehazing constraints and denoising constraints, the practicality of the dehazing model and the clarity of the dehazed image are improved.

[0028] According to an embodiment of the present invention, a deep learning image dehazing method based on self-supervised single image is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] This embodiment provides a deep learning-based image dehazing method based on self-supervised single-image processing, which can be used in the aforementioned computer system. Figure 2 This is a flowchart of a deep learning image dehazing method based on self-supervised single-image processing according to an embodiment of the present invention, as shown below. Figure 2 As shown, the process includes the following steps: Step S201: Obtain the fog map to be processed and perform self-supervised preprocessing on the fog map to be processed to obtain the self-supervised fog map.

[0030] Specifically, the fog image to be processed is an image captured by an image acquisition device (mobile phone, camera, etc.) under adverse weather conditions (foggy days, rainy days, etc.). Due to the weather, the directly captured image is blurry. Only one fog image needs to be acquired, and self-supervised preprocessing is performed on the fog image to obtain multiple self-supervised fog images for model training. The self-supervised preprocessing process includes, but is not limited to, image resampling, upsampling, and data augmentation.

[0031] Step S202: Design a loss function based on fog constraints and denoising constraints, and construct a self-supervised network model.

[0032] Specifically, a neural network model is constructed as a self-supervised network model, and a loss function corresponding to the neural network model is designed. To ensure the dehazing effect, the loss function must include a dehazing constraint; to ensure image clarity and reduce noise, the loss function must include a denoising constraint (Noise2Noise, N2N). In this embodiment, the Retinex algorithm is used as the dehazing constraint during training, and the denoising constraint is designed based on the principle of denoising using the zero-mean characteristic of noise. The dehazing constraint and the denoising constraint are superimposed to obtain the loss function of the self-supervised network model.

[0033] Step S203: Using the self-supervised fog map, the self-supervised network model is iteratively trained based on the loss function to obtain the dehazing model and the reconstructed dehazing image. Specifically, the self-supervised hazy image is used as a training sample and input into the self-supervised network model for iterative training. The loss function is used as a training constraint. Each time, the loss function is calculated based on the dehazed image output by the self-supervised network model until the dehazed image simultaneously satisfies the dehazing constraint and the denoising constraint in the loss function. The training ends at this point. The parameters of the self-supervised network model are the optimized parameters after training, and the corresponding self-supervised network model is the dehazing model (Self-Supervised Hazy-to-Hazy Network, SH2H).

[0034] The model iterative training process and the image reconstruction process are carried out simultaneously. During the training of the self-supervised network model, each iteration yields a clearer image. When the loss function of the self-supervised network model converges, it indicates that the obtained image satisfies the dehazing and denoising constraints, and the original fog image to be processed is reconstructed into a dehazed image.

[0035] The deep learning image dehazing method based on self-supervised training of a single image provided in this embodiment designs a loss function based on dehazing constraints and denoising constraints. The dehazing constraints realize image dehazing mapping, and the denoising constraints eliminate noise. High-quality image dehazing reconstruction is achieved through self-supervised training based on a single fog map. It overcomes the shortcomings of traditional deep learning, such as dependence on large-scale datasets and poor generalization ability. It has the characteristics of high image restoration quality, strong generalization ability, and strong practicality.

[0036] This embodiment provides a deep learning-based image dehazing method based on self-supervised single-image processing, which can be used in the aforementioned computer system. Figure 3 This is a flowchart of a deep learning image dehazing method based on self-supervised single-image processing according to an embodiment of the present invention, as shown below. Figure 3 As shown, the process includes the following steps: Step S301: Obtain the fog map to be processed and perform self-supervised preprocessing on the fog map to be processed to obtain a self-supervised fog map.

[0037] Specifically, step S301 includes: Step S3011: Diagonal resampling is performed on the fog image to be processed to obtain two mutually supervised images.

[0038] Specifically, the principle of N2N denoising constraint is based on the zero-mean characteristic of noise to achieve denoising. Therefore, N2N usually requires two noisy, independent images of the same scene to be used as mutual supervision data. However, taking two images means an increase in sampling time, which is not conducive to real-time image processing. Therefore, the fog image to be processed in this embodiment is a single image. Figure 4 As shown, the fog image to be processed is resampled diagonally according to pixels to obtain two mutual supervision images, with the green square and the blue square being a mutual supervision image respectively.

[0039] Step S3012: The mutual supervision image is upsampled using the Fourier domain zero-filling method so that the size of the mutual supervision image is the same as that of the fog image to be processed.

[0040] Specifically, the mutual supervision image obtained after diagonal resampling suffers a resolution loss of twice that of the original fog image to be processed, due to a two-fold downsampling. Therefore, it is necessary to upsample the mutual supervision image using the Fourier domain zero-padding method to restore the image size and make the size of the mutual supervision image the same as that of the fog image to be processed. The process of upsampling using the Fourier domain is a mature existing technology and will not be elaborated here.

[0041] Step S3013: Perform data augmentation based on the mutual supervised images to obtain a self-supervised fog map, which includes the image sets corresponding to the two mutual supervised images respectively.

[0042] Specifically, after upsampling using the Fourier domain, two mutually supervised images of the same size as the original fog image to be processed are obtained. For model training, two images are too few. To expand the amount of training data for the network, data augmentation can be performed by random patch transformation, image flipping and rotation, etc. This process is a mature existing technology and will not be elaborated here.

[0043] In some optional implementations, step S3011 above includes: Step a1: Based on the preset division principle, divide multiple adjacent pixels in the fog image to be processed into a square unit, and traverse each pixel in the fog image to be processed to obtain multiple square units.

[0044] Specifically, to ensure the reasonable effectiveness of the mutual supervision images, a diagonal resampling method is adopted to divide multiple adjacent pixels in the fog image to be processed into a square unit. For example, four adjacent pixels in a 2×2 square or nine adjacent pixels in a 3×3 square are used as examples, but are not limited to this. The entire fog image to be processed is traversed and divided without overlap or omission, resulting in a fog image composed of multiple square units.

[0045] Step a2: Calculate the average pixel value on the diagonal of each square unit, and use it as the first new pixel and the second new pixel, respectively.

[0046] Specifically, taking a 2×2 square of four adjacent pixels as an example, each square unit includes two diagonals. The average value of the pixels along each diagonal is calculated to obtain the first new pixel and the second new pixel. The first new pixel and the second new pixel each include the average value of the pixels along the same diagonal. For example, if the four pixels are (x, y), (x+1, y), (x, y+1), and (x+1, y+1), and the pixels along the two diagonals are (x, y), (x+1, y+1) and (x+1, y), (x, y+1), respectively, the average value of each pixel is calculated to obtain the first new pixel. Second new pixel .

[0047] Step a3: Use the first new pixel and the second new pixel to construct two mutually supervised images respectively.

[0048] Specifically, the first and second new pixels on the same diagonal line constitute two mutually supervised images, for example... Figure 4 The new pixels corresponding to the green square and the blue square in the middle form a mutual supervision image. The two mutual supervision images share the same details, but the noise is relatively independent.

[0049] The deep learning image dehazing method based on single-image self-supervision provided in this embodiment obtains two mutually supervised images through diagonal resampling, matches the size of the original fog map using Fourier domain zero-padding upsampling, and then generates a self-supervised fog map through data augmentation. This generates image data that meets the requirements of self-supervised training and has suitable detail and noise characteristics, providing high-quality samples for subsequent dehazing model training.

[0050] Step S302: Design a loss function based on dehazing and denoising constraints, and construct a self-supervised network model.

[0051] Specifically, the self-supervised fog map includes a first self-supervised image and a second self-supervised image. The first self-supervised image and the second self-supervised image have the same image content and their noise distributions are mutually exclusive. The dehazing process based on the dehazing constraint part includes: Step S3021: The second self-supervised image is dehazed using the Retinex algorithm to obtain the second dehazed image.

[0052] Specifically, self-supervised network models do not require large-scale datasets for training; they only need a single fog map for self-supervised training. Therefore, the design of the loss function directly determines the performance of the network model.

[0053] The self-supervised fog map consists of two image sets corresponding to two mutually supervised images. The two mutually supervised images are taken as the first self-supervised image and the second self-supervised image, respectively. The second self-supervised image is then dehazed using the Retinex algorithm to obtain the second dehazed image.

[0054] Step S3022: Substitute the second dehazed image into the visual perception model to obtain the second hazy image.

[0055] Specifically, the second dehazed image is substituted into the visual perception model: The corresponding second foggy image is obtained, where, This indicates the second foggy image. This indicates that it is derived from Gaussian filtering in the Retinex algorithm. This represents the second dehazed image.

[0056] Step S3023: Based on minimizing the distance between the second foggy image and the first self-supervised image, construct the defogging constraint part in the loss function, and based on the defogging constraint part, use the self-supervised network model to achieve defogging.

[0057] Specifically, the dehazing constraint part of the loss function is designed based on the second foggy image and the first self-supervised image, and its expression is as follows: (1) in, Indicates constraint weights. This indicates the weight of the defogging constraint. This represents the first self-supervised image. This indicates the second foggy image. .

[0058] In this embodiment, to clarify the principles of dehazing and denoising, dehazing and denoising are explained separately. In actual processing, the dehazing constraint part and the denoising constraint part in the loss function work simultaneously. The image processed by the self-supervised network model is the reconstructed image after dehazing and denoising.

[0059] In some alternative implementations, the process of partial line-of-sight denoising based on denoising constraints includes: Step S3024: Input the first self-supervised image and the second self-supervised image into the self-supervised network model respectively to obtain the first network output image and the second network output image.

[0060] Step S3025: Assuming the noise median of the first network output image and the second output image is 0, construct the denoising constraint part of the loss function with the goal of minimizing the distance between the first network output image and the second self-supervised image.

[0061] Specifically, the denoising constraint part of the loss function is designed based on the first and second self-supervised images, and its expression is as follows: (2) Specifically, This represents the first self-supervised image. This represents the second self-supervised image. This represents the first output image of the self-supervised network model after the first self-supervised image is input. This represents the second self-supervised image input to the self-supervised network model, and its output image. Indicates constraint weights. This indicates the weight of the denoising constraint.

[0062] By superimposing the dehazing constraint and the denoising constraint, the loss function of the self-supervised network model is obtained: (3) The dehazing constraint part can ensure that the network model converges in accordance with the traditional dehazing model, which can meet the needs of the dehazing task; the denoising constraint part can effectively remove image noise and improve the clarity of the processed image.

[0063] Step S3026: Denoising is achieved using the variational method based on the denoising constraint part of the self-supervised network model.

[0064] Specifically, the principle of denoising using the denoising constraint part is as follows: Assuming two mutually supervised images consist of a sharp image and independent noise, the first self-supervised image includes a sharp image and a first noise: The second self-supervised image consists of a sharp image and a second noise: ,in, This represents the first self-supervised image. This represents the second self-supervised image. Indicates a clear image. and denoted as independent and identically distributed noise corresponding to the first self-supervised image and the second self-supervised image, respectively.

[0065] The objective of the denoising constraint part of the loss function is to minimize the distance between the first network output image and the second self-supervised image, expressed by the formula: (4) in, This represents the output image of the first network. This represents the second self-supervised image. This represents the model parameters. For any pixel location, the problem can be simplified to: (5) in, This represents the output of the first network at the corresponding pixel position.

[0066] Based on the variational method, the optimal solution for the denoising loss function is determined to be the conditional median: (6) in, This represents the first self-supervised image. This represents the second self-supervised image. Let c represent the optimal solution.

[0067] Assuming the median of the second noise is 0, the splitting result of the second self-supervised image... Substituting into formula (6) and combining it with noise independence, we get: (7) Because N2N denoising theory is based on the zero-mean characteristic of noise ( However, considering the need to avoid excessive smoothing of edge information and to resist outlier noise, the denoising constraint part of the loss function based on the zero median assumption is used. Perform iterative optimization, assuming the median of the second noise. We can obtain: (8) in, This represents the output image after noise reduction processing. Since the image represents a clear image, a clear image can be obtained by denoising through a self-supervised network model.

[0068] The deep learning-based image dehazing method based on single-image self-supervision provided in this embodiment combines dehazing and visual perception model operations to design a loss function. It incorporates dehazing constraints to ensure the direction of dehazing and also utilizes denoising constraints. Based on image segmentation, distance minimization, variational methods, and noise independence, it achieves denoising under the assumption that the noise median is 0. This allows the self-supervised network model to simultaneously consider the accuracy of dehazing and the effectiveness of denoising, improving the model's processing quality of foggy images and enhancing the clarity and purity of the dehazed image.

[0069] Step S303: Using the self-supervised fog map, the self-supervised network model is iteratively trained based on the loss function to obtain the dehazing model and the reconstructed dehazing image.

[0070] In some optional implementations, the self-supervised network model is a convolutional neural network architecture, which includes an encoder module, a decoder module, and a skip module. The encoder module is connected to a nonlinear activation unit and a max pooling unit in sequence, and the decoder module is connected to a nonlinear activation unit and a bilinear interpolation unit in sequence.

[0071] Specifically, a self-supervised network model for forward dehazing is implemented using a convolutional neural network model (U-net), comprising: an encoder module, a decoder module, and four skip connections. Each encoder block contains two 3×3 convolutional layers, followed by a non-linear activation unit (Leaky ReLU), and then a 2×2 max pooling operation with a stride of two. Conversely, each decoder block contains two 3×3 convolutional layers, followed by a Leaky ReLU, and then a two-dimensional bilinear interpolation operation.

[0072] Specifically, step S303 includes: Step S3031: Input the self-supervised fog map into the self-supervised network model for iterative training, and calculate the corresponding loss function each time.

[0073] Step S3032 continues until the loss function converges or satisfies the preset conditions, resulting in a well-trained dehazing model.

[0074] Specifically, the self-supervised fog map includes an image set corresponding to two mutually supervised images. The two mutually supervised images are used as the input image and ground truth image of the network model, respectively, to form a paired training data pair to train the self-supervised network model. The training convergence is constrained by the loss function of this embodiment. After multiple rounds of training, the network completes the training and can output a clear defogging image.

[0075] This embodiment provides a deep learning-based image dehazing method based on self-supervised single-image processing. It employs a convolutional neural network architecture to construct a self-supervised network model. The encoder module combines nonlinear activation and max pooling, while the decoder module uses nonlinear activation and bilinear interpolation. This effectively extracts fog map features and accurately restores image details. Utilizing the self-supervised fog map, the model is iteratively trained based on a loss function until the loss function converges. This allows the trained dehazing model to accurately learn dehazing rules, improving the dehazing effect and resulting in a clearer dehazed image.

[0076] In one specific embodiment, the dehazing method provided in this embodiment is used to process a hazy image to restore its clarity, such as... Figure 5 As shown, images taken in foggy weather can be directly input into a trained dehazing model to reconstruct clear, dehazed scene images. The dehazing model does not require training on a large-scale paired dataset; it only needs to extract the diagonal pixels of a single fog image and then perform Fourier domain upsampling to reconstruct a clear, dehazed scene image. Figure 1 The image is split into two identical fog images. These two newly generated fog images are then used for self-supervised training. Through iterative iterations, the dehazing model can reconstruct a clear image. The specific process includes: (a) Dehazing workflow: Hazy images captured by the camera can be directly input into the SH2H network to achieve image dehazing.

[0077] (b) Self-supervised data generation: The number of extracted pixels is The diagonal pixels of a single-frame fog image are decomposed into Two images of pixels are upsampled in the Fourier domain and then enlarged to the same size. Two images of pixels.

[0078] (c) Self-supervised network model training process: A U-Net network is trained in a self-supervised manner using two newly generated fog images. Its loss function includes two parts: defogging constraint Retinex_loss and denoising constraint N2N_loss.

[0079] (d) Mathematical expression of loss function: including dehazing constraint Retinex and denoising constraint N2N.

[0080] (e) Experimental results: In foggy scenes, the results of the dehazing reconstruction of images taken by the image acquisition device by the trained dehazing model are compared with the original fog image.

[0081] The dehazing method provided in this embodiment outperforms traditional dehazing optimization algorithms based on atmospheric scattering models in terms of reconstruction results. SH2H has demonstrated its significant image dehazing capabilities on publicly available synthetic fog image datasets, real-world remote sensing fog image datasets, and real-world fog image datasets taken by smartphones. It also proves that the algorithm has excellent versatility and generalization ability, broad adaptability, and is suitable for various imaging devices and scenarios. SH2H exhibits high engineering application potential and can help promote the development of various computer vision tasks, such as object detection, tracking, and environmental monitoring.

[0082] This embodiment also provides a deep learning-based image dehazing device based on single-image self-supervision. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0083] This embodiment provides a deep learning-based image dehazing device based on self-supervised single-image processing, such as... Figure 6 As shown, it includes: Image preprocessing module 601 is used to acquire the fog image to be processed and perform self-supervised preprocessing on the fog image to obtain a self-supervised fog image.

[0084] The loss function design module 602 is used to design a loss function based on dehazing and denoising constraints, and to build a self-supervised network model.

[0085] The model training and dehazing module 603 is used to iteratively train the self-supervised network model based on the loss function using the self-supervised fog map to obtain the dehazing model and the reconstructed dehazing image.

[0086] In some alternative implementations, the image preprocessing module 601 includes: The resampling unit is used to resample the fog image to be processed diagonally to obtain two mutually supervised images.

[0087] The upsampling unit is used to upsample the mutual supervision image using the Fourier domain zero-padding method, so that the size of the mutual supervision image is the same as that of the fog image to be processed.

[0088] The data augmentation unit is used to perform data augmentation based on the mutually supervised images to obtain a self-supervised fog map, which includes the image sets corresponding to the two mutually supervised images.

[0089] In some alternative implementations, the resampling unit includes: The unit division sub-unit is used to divide multiple adjacent pixels in the fog image to be processed into a square unit based on a preset division principle. Multiple square units are obtained by traversing each pixel in the fog image to be processed.

[0090] The new pixel calculation subunit is used to calculate the average pixel value on the diagonal of each square unit, which are respectively used as the first new pixel and the second new pixel.

[0091] The mutual supervision image determination unit is used to construct two mutual supervision images using the first new pixel and the second new pixel respectively.

[0092] In some alternative implementations, the loss function design module 602 includes: The first dehazing processing unit is used to perform dehazing processing on the second self-supervised image based on the Retinex algorithm to obtain the second dehazed image.

[0093] The second foggy image determination unit is used to substitute the second defogging image into the visual perception model to obtain the second foggy image.

[0094] The dehazing constraint design unit is used to construct the dehazing constraint part in the loss function based on minimizing the distance between the second foggy image and the first self-supervised image, and to achieve dehazing based on the dehazing constraint part of the self-supervised network model.

[0095] The first denoising processing unit is used to input the first self-supervised image and the second self-supervised image into the self-supervised network model respectively to obtain the first network output image and the second network output image.

[0096] The denoising constraint design unit is used to construct the denoising constraint part of the loss function by assuming that the noise median of the first network output image and the second output image is 0, with the goal of minimizing the distance between the first network output image and the second self-supervised image.

[0097] The denoising unit is used to achieve denoising based on the denoising constraint part of the self-supervised network model using the variational method.

[0098] In some alternative implementations, the model training and dehazing module 603 includes: The iterative training unit is used to input the self-supervised fog map into the self-supervised network model for iterative training, calculating the corresponding loss function each time.

[0099] The training termination unit is used until the loss function converges or a preset condition is met, resulting in a well-trained dehazing model.

[0100] The image dehazing device based on self-supervised deep learning for single images provided in this invention can execute the image dehazing method based on self-supervised deep learning for single images provided in any embodiment of this invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the various modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0101] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0102] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0103] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0104] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the single-image self-supervised deep learning image dehazing method of the embodiments of the present invention.

[0105] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0106] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the deep learning image dehazing method based on single-image self-supervised learning shown in the above embodiments is implemented.

[0107] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A deep learning-based image dehazing method based on self-supervised single-image processing, characterized in that, The method includes: Obtain the fog image to be processed, and perform self-supervised preprocessing on the fog image to obtain a self-supervised fog image; A loss function is designed based on a combination of defogging and denoising constraints, and a self-supervised network model is constructed. Using the self-supervised fog map, the self-supervised network model is iteratively trained based on the loss function to obtain the defogging model and the reconstructed defogging image; The self-supervised fog map includes a first self-supervised image and a second self-supervised image, wherein the first self-supervised image and the second self-supervised image have the same image content and their noises are mutually oppositely distributed; the defogging process based on the defogging constraint part includes: performing defogging processing on the second self-supervised image using the Retinex algorithm to obtain a second defogging image; substituting the second defogging image into a visual perception model to obtain a second foggy image; constructing the defogging constraint part in the loss function based on minimizing the distance between the second foggy image and the first self-supervised image as the objective, and realizing defogging based on the defogging constraint part of the self-supervised network model.

2. The method according to claim 1, characterized in that, The fog map to be processed is subjected to self-supervised preprocessing to obtain a self-supervised fog map, including: The fog image to be processed is resampled diagonally to obtain two mutually supervised images; The mutual supervision image is upsampled using the Fourier domain zero-filling method so that the size of the mutual supervision image is the same as that of the fog image to be processed; Data augmentation is performed based on the mutual supervised images to obtain a self-supervised fog map, which includes image sets corresponding to the two mutual supervised images respectively.

3. The method according to claim 2, characterized in that, The fog image to be processed is resampled diagonally to obtain two mutually supervised images, including: Based on a preset division principle, multiple adjacent pixels in the fog image to be processed are divided into a square unit, and multiple square units are obtained by traversing each pixel in the fog image to be processed. Calculate the average pixel value on the diagonal of each square unit, and use it as the first new pixel and the second new pixel respectively; Two mutually supervised images are constructed using the first new pixel and the second new pixel, respectively.

4. The method according to claim 1, characterized in that, The process of partial line-of-sight denoising based on denoising constraints includes: The first self-supervised image and the second self-supervised image are respectively input into the self-supervised network model to obtain the first network output image and the second network output image; Assuming the noise median of the first network output image and the second output image is 0, the denoising constraint part of the loss function is constructed with the goal of minimizing the distance between the first network output image and the second self-supervised image. Denoising is achieved by using the variational method and the denoising constraint part of the self-supervised network model.

5. The method according to claim 1, characterized in that, The self-supervised network model is a convolutional neural network architecture. Constructing the self-supervised network model includes: Construct a convolutional neural network architecture, which includes: an encoder module, a decoder module, and a hopping module, wherein... The encoder module is connected in sequence to the nonlinear activation unit and the max pooling unit, and the decoder module is connected in sequence to the nonlinear activation unit and the bilinear interpolation unit.

6. The method according to claim 5, characterized in that, Using the self-supervised fog map, the self-supervised network model is trained based on the loss function to obtain a defogging model, including: The self-supervised fog map is input into the self-supervised network model for iterative training, and the corresponding loss function is calculated each time. The trained dehazing model is obtained when the loss function converges or satisfies the preset conditions.

7. A deep learning-based image dehazing device based on self-supervised single-image processing, characterized in that, The device includes: An image preprocessing module is used to acquire a fog image to be processed and to perform self-supervised preprocessing on the fog image to be processed to obtain a self-supervised fog image. The loss function design module is used to design loss functions based on dehazing and denoising constraints, and to build a self-supervised network model. The model training and dehazing module is used to iteratively train the self-supervised network model based on the loss function using the self-supervised fog map to obtain the dehazing model and the reconstructed dehazing image. The self-supervised fog map includes a first self-supervised image and a second self-supervised image, wherein the first self-supervised image and the second self-supervised image have the same image content and their noises are mutually oppositely distributed; the defogging process based on the defogging constraint part includes: performing defogging processing on the second self-supervised image using the Retinex algorithm to obtain a second defogging image; substituting the second defogging image into a visual perception model to obtain a second foggy image; constructing the defogging constraint part in the loss function based on minimizing the distance between the second foggy image and the first self-supervised image as the objective, and realizing defogging based on the defogging constraint part of the self-supervised network model.

8. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 6.

Images