Method for training defogging model based on unpaired images
By generating synthetic foggy images and establishing a total loss function for dual-branch training, the problem that the existing unpaired image dehazing model performs poorly in real haze scenes is solved, achieving a more stable dehazing effect and stronger haze feature learning ability.
Patent Information
- Application Number
- CN202510769047.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-09-26
AI Technical Summary
The existing dehazing model based on unpaired images does not perform well in real haze scenes, and the existing learning framework is unstable in training, resulting in a decrease in dehazing effect.
By obtaining multiple foggy and fog-free images, using the depth estimation model and atmospheric scattering model to generate a synthetic foggy image, establishing a total loss function, and performing two-branch joint training, we can learn the haze feature relationship between fog-free images, foggy images, and synthetic foggy images, thereby improving the stability of the defogging model and the defogging effect.
The defogging model has improved its defogging effect on real haze scenes, enhanced the training stability of the model, reduced the problem of degradation of the defogging effect, and can effectively learn the characteristics of haze of different concentrations and improve the defogging ability.
Smart Images
Figure CN120707438A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of computer vision technology and image processing technology, and more specifically, to a method for training a defogging model based on unpaired images. Background Art
[0002] The low-quality images caused by haze degradation not only affect visual perception but also interfere with computer vision models such as object detection and scene segmentation. Therefore, neural network-based dehazing models are required for dehazing. Currently, most neural network-based dehazing models are trained using datasets of paired foggy and fog-free images. Since paired foggy and fog-free images are rare in real scenes, most are trained using synthetic datasets. This involves using a fogging model to generate fogged images that are paired with fog-free images. However, due to the discrepancy between synthetic datasets and real haze, the actual dehazing performance of the trained dehazing models may be reduced. Therefore, some methods have begun to emerge that use unpaired real-world clean and foggy images to train dehazing models separately.
[0003] Existing unpaired image dehazing models are mostly trained using learning frameworks such as CycleGAN or contrastive learning. Because these frameworks require complex sampling strategies, generators, and discriminators, they suffer from training instability, which can lead to reduced dehazing performance. Furthermore, even after dehazing models are trained using existing unpaired image learning frameworks, they still perform poorly in real-world haze scenes. Summary of the Invention
[0004] In response to the above-mentioned defects in the prior art, the present application provides a method for training a defogging model based on unpaired images, aiming to solve the problem that after the existing defogging model is trained based on a learning framework based on unpaired images, the defogging effect on real haze scenes is still poor.
[0005] In a first aspect, the present application provides a method for training a defogging model based on unpaired images, comprising: Obtain multiple foggy images and multiple fog-free images; Obtaining a first synthetic foggy image based on the fog-free image, and training a defogging model based on each fog-free image and the first synthetic foggy image corresponding to the fog-free image to obtain a pre-trained defogging model; Obtaining a second synthetic foggy image based on the foggy image, and inputting the foggy image and the second synthetic foggy image into a pre-trained defogging model respectively to obtain a first defogging image corresponding to the foggy image and a second defogging image corresponding to the second synthetic foggy image; Establishing a total loss function based on the image difference between each first dehazed image and the second dehazed image corresponding to the first dehazed image; Based on the total loss function, loss calculation is performed on each fog-free image and the first synthetic foggy image, and each foggy image and the second synthetic foggy image, and the pre-trained defogging model is trained based on the results of the loss calculation.
[0006] Furthermore, obtaining a first synthetic foggy image based on the fog-free image includes: A first depth map corresponding to the fog-free image is obtained based on the depth estimation model, and based on a first predicted depth matrix corresponding to the first depth map, the fog-free image is fogged using an atmospheric scattering model to obtain a first synthetic foggy image.
[0007] Among them, since the atmospheric scattering model needs to obtain the depth value of each pixel point of the image when adding fog to the fog-free image, the depth estimation model is used to obtain the first depth map corresponding to the fog-free image, and a stable and reliable first predicted depth matrix can be obtained as the approximate depth in the atmospheric scattering model adding fog process.
[0008] Furthermore, the fog-free image is fogged using the atmospheric scattering model, including: Establishing a first atmospheric light value interval, randomly selecting any value from the first atmospheric scattering coefficient value interval, and constructing a first atmospheric light matrix corresponding to the first predicted depth matrix; Establishing a first atmospheric scattering coefficient numerical interval, randomly selecting any value from the first atmospheric scattering coefficient numerical interval, and constructing a first atmospheric scattering coefficient matrix corresponding to the first predicted depth matrix; Based on the first predicted depth matrix, the first atmospheric scattering coefficient matrix and the first atmospheric light matrix, the fog-free image is fogged using the atmospheric scattering model.
[0009] Furthermore, a defogging model is trained based on each fog-free image and a first synthetic foggy image corresponding to the fog-free image, including: Based on each fog-free image and the first synthetic foggy image corresponding to the fog-free image, a first loss function is established, and the loss of the defogging result image of the fog-free image and the first synthetic foggy image corresponding to the fog-free image is calculated based on the first loss function, and the parameters of the defogging model are trained and updated based on the loss calculation result.
[0010] Furthermore, the first loss function is implemented by the following formula: , in, represents the first loss function, It represents the defogging result image of the first synthetic foggy image. represents a fog-free image, represents the L1 loss function.
[0011] Among them, the first loss function is used to pre-train the defogging model so that the defogging model can effectively learn the feature differences between the haze-free image and the synthetic haze image, so that the defogging model has preliminary defogging capabilities.
[0012] Further, obtaining a second synthetic foggy image based on the foggy image includes: A second depth map corresponding to the foggy image is obtained based on the depth estimation model, and based on a second predicted depth matrix corresponding to the second depth map, the foggy image is fogged using a target prior estimation algorithm and an atmospheric scattering model to obtain a second synthetic foggy image.
[0013] Furthermore, the foggy image is fogged using the target prior estimation algorithm and the atmospheric scattering model, including: Based on the second predicted depth matrix, a second atmospheric light matrix corresponding to the second predicted depth matrix is obtained using a target prior estimation algorithm; Establishing a second atmospheric scattering coefficient numerical interval, randomly selecting any value from the second atmospheric scattering coefficient numerical interval, and constructing a second atmospheric scattering coefficient matrix corresponding to the second predicted depth matrix; Based on the second predicted depth matrix, the second atmospheric scattering coefficient matrix and the second atmospheric light matrix, the foggy image is fogged using the atmospheric scattering model.
[0014] Among them, the second atmospheric light matrix is estimated by the target prior estimation algorithm instead of using the second atmospheric light matrix generated by random depth values, so that fog can be added again based on the haze concentration of the foggy image, thereby obtaining a second synthetic foggy image.
[0015] Furthermore, a total loss function is established based on the image difference between each first dehazed image and the second dehazed image corresponding to the first dehazed image, including: Establishing a third loss function based on image feature deviations between each first dehazed image and the second dehazed image corresponding to the first dehazed image; Establishing a depth loss function based on the image depth deviations of each foggy image, the first dehazed image corresponding to the foggy image, and the second dehazed image; A semantic loss function is established based on the image semantic deviations of each foggy image, the first dehazed image corresponding to the foggy image, and the second dehazed image, and a second loss function is established based on the semantic loss function, the depth loss function, and the third loss function; A total loss function is established based on the first loss function and the second loss function.
[0016] Among them, by training the dehazing model through the third loss function, semantic loss function and depth loss function, the dehazing model can learn the feature connection between the foggy image and the second synthetic foggy image, thereby improving the dehazing model's feature extraction ability for haze of different concentrations, and thus improving the dehazing ability of the dehazing model.
[0017] Furthermore, the second loss function is implemented by the following formula: ; ; ; ; in, represents the second loss function, represents the third loss function, represents the depth loss function, represents the semantic loss function, represents the depth balance parameter, represents the semantic balance parameter, represents the first dehazed image, represents the second dehazed image, Indicates a foggy image. represents the L1 loss function, represents the depth estimation operation, represents the semantic extraction operation, Indicates KL divergence calculation.
[0018] Furthermore, based on the total loss function, loss calculation is performed on each fog-free image and the first synthetic foggy image, and each foggy image and the second synthetic foggy image, respectively, and the pre-trained dehazing model is trained based on the loss calculation results, including: The first synthetic foggy image corresponding to each fog-free image is input into the first pre-trained model, and the second synthetic foggy image corresponding to each foggy image and the foggy image is synchronously input into two pre-trained defogging models respectively; Calculate a first loss of the first synthetic foggy image corresponding to the fog-free image and the fog-free image based on the first loss function, and calculate a second loss of the second synthetic foggy image corresponding to the foggy image and the foggy image based on the second loss function, The pre-trained dehazing model is trained based on the calculation results of the first loss and the second damage.
[0019] Among them, the fog-free image and the first synthetic foggy image, each foggy image and the second synthetic foggy image are simultaneously input into the feature input channel of the dehazing model, and synchronous training is performed through the total loss function. This is to enable the dehazing model to learn the haze feature relationship between the fog-free image, the foggy image and the synthetic foggy image, and then effectively learn the haze features of different concentrations close to the real scene, thereby further improving the dehazing ability of the dehazing model.
[0020] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art: 1. The training method of the present invention does not require the design of a generator and a discriminator with a complex structure for training separately from the structure of the defogging model. It can improve the training stability of the defogging model and reduce the problem of decreased defogging effect of the defogging model during training.
[0021] 2. The present invention uses the designed total loss function to perform dual-branch joint training on the dehazing model using fog-free images and foggy images respectively. The dehazing model corresponding to the fog-free image branch is preliminarily trained using the first loss function, and then the dehazing models corresponding to the fog-free branch and the foggy branch are synchronously trained using the total loss function. This enables the dehazing model to learn the connection between the synthetic foggy image and the fog-free image, and between the foggy image and the synthetic foggy image, thereby learning the haze characteristics close to the real haze scene, and ultimately effectively improving the dehazing effect of the dehazing model. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in this application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0023] Figure 1 This is a flow chart of a method for training a defogging model based on unpaired images provided in an embodiment of the present application; Figure 2 This is another flowchart of the method for training a defogging model based on unpaired images provided in an embodiment of the present application; Figure 3 Schematic diagram of the structure of the defogging model provided in the embodiment of the present application; Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.
[0025] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application. Different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, even though the embodiment may not be clearly described in the following text.
[0026] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the elements described without departing from the scope of the present application. Various examples may appropriately omit, replace, or add various processes or components. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted, or combined. In addition, features described in some examples may be combined in other examples.
[0027] like Figure 1 As shown, Figure 1 : is a flow chart of a method for training a defogging model based on unpaired images provided in an embodiment of the present application, such as Figure 1 As shown, the method comprises at least the following steps: S101. Acquire multiple foggy images and multiple fog-free images.
[0028] The execution entity of this method can be a computer processor, an embedded device, or a processor of a cloud service platform. The foggy and fog-free images used in this embodiment are from a target training dataset, where there are no paired images for either the foggy or fog-free images. The target training dataset can be one or more datasets for haze scenes without paired images, and the RESIDE-unpaired dataset from the public RESIDE (Realistic Single Image Dehazing) dataset can be used. After training the dehazing model, the I-HAZE dataset or the RTTS dataset can also be used as a test set to evaluate the dehazing effect.
[0029] S102: Obtain a first synthetic foggy image based on the fog-free image, and train a defogging model based on each fog-free image and the first synthetic foggy image corresponding to the fog-free image to obtain a pre-trained defogging model.
[0030] In the embodiments of this application, Figure 2As shown, the embodiment of the present application adopts two branches to train the foggy image and the fog-free image. For the fog-free image branch, the fog-free image is fogged to obtain a paired first synthetic foggy image, and then the defogging model is trained by using the first synthetic foggy image that is paired with each fog-free image.
[0031] Among them, the dehazing model can use deep learning dehazing networks such as NAFNet and DehazeFormer. Figure 3 As shown, the dehazing model in this embodiment specifically uses a U-Net design with a three-scale input and output structure. The DehazeFormer deep learning network serves as the underlying structure between the lowest-scale input channels to mitigate boundary artifacts between the input and output images. Furthermore, multiple NAFNet feature enhancement modules are deployed between input channels at other scales to reduce boundary artifacts between the input and output images. This dehazing model also boasts the advantages of a small model size and low computing power consumption, enabling simultaneous training of multiple sets of images using multiple sets of feature channels, thus meeting multi-branch training requirements.
[0032] In one embodiment, obtaining a first synthetic foggy image based on the fog-free image includes: A first depth map corresponding to the fog-free image is obtained based on the depth estimation model, and based on a first predicted depth matrix corresponding to the first depth map, the fog-free image is fogged using an atmospheric scattering model to obtain a first synthetic foggy image.
[0033] In an embodiment of the present application, the DepthAnything model can be selected as the depth estimation model. This is because when the atmospheric scattering model adds fog to the fog-free image, it is necessary to obtain the depth value of each pixel of the image, and the first predicted depth matrix is a numerical matrix of the depth values of each pixel. Since the DepthAnything model has strong generalization capabilities and can take into account the estimation of both the relative depth and absolute depth of the image, it can stably and accurately estimate the depth matrix of the input image under various concentrations of haze, thereby improving the image quality of the first synthetic foggy image obtained by adding fog to the fog-free image.
[0034] In addition, in this embodiment, fogging of the image is completed by an atmospheric scattering model. The fogging process of the atmospheric scattering model is as follows:
[0035] in, represents the image after adding fog, represents the image before fogging, represents the transmission diagram, and denote the atmospheric scattering coefficient and atmospheric light, respectively. Indicates depth.
[0036] In one possible implementation, fogging is performed on a fog-free image using an atmospheric scattering model, including: Establishing a first atmospheric light value interval, randomly selecting any value from the first atmospheric scattering coefficient value interval, and constructing a first atmospheric light matrix corresponding to the first predicted depth matrix; Establishing a first atmospheric scattering coefficient numerical interval, randomly selecting any value from the first atmospheric scattering coefficient numerical interval, and constructing a first atmospheric scattering coefficient matrix corresponding to the first predicted depth matrix; Based on the first predicted depth matrix, the first atmospheric scattering coefficient matrix and the first atmospheric light matrix, the fog-free image is fogged using the atmospheric scattering model.
[0037] In an embodiment of the present application, since the defogging model is pre-trained directly through the fog-free image and the first synthetic foggy image, the values in the first atmospheric light matrix and the first atmospheric scattering coefficient matrix in the fogging process of the atmospheric scattering model need to be completely random to improve the generalization ability of the pre-trained model.
[0038] In one possible implementation, training a defogging model based on each fog-free image and a first synthetic foggy image corresponding to the fog-free image includes: Based on each fog-free image and the first synthetic foggy image corresponding to the fog-free image, a first loss function is established, and the loss of the defogging result image of the fog-free image and the first synthetic foggy image corresponding to the fog-free image is calculated based on the first loss function, and the parameters of the defogging model are trained and updated based on the loss calculation result.
[0039] In the embodiments of this application, Figure 2 As shown in the fog-free branch, the fog model will predict the first synthetic foggy image input to obtain a defogging result image with the same scale as the fog-free image. Then, the first loss function can be used to calculate the feature deviation between the defogging result image and the fog-free image, thereby realizing pre-training of the defogging model.
[0040] In one embodiment, the first loss function is implemented by the following formula: , in, represents the first loss function, It represents the defogging result image of the first synthetic foggy image. represents a fog-free image, represents the L1 loss function.
[0041] In the embodiment of the present application, the first loss function uses the L1 loss function because The L1 loss function has strong robustness to outliers, detail recovery ability and computational efficiency. For the direct comparison of image features such as the haze-free image and the first synthetic haze image, the L1 loss function is more suitable.
[0042] S103. Obtain a second synthetic foggy image based on the foggy image, and input the foggy image and the second synthetic foggy image into a pre-trained defogging model respectively to obtain a first defogged image corresponding to the foggy image and a second defogged image corresponding to the second synthetic foggy image.
[0043] In the embodiments of this application, Figure 2 As shown, the first dehazed image and the second dehazed image are obtained through the dehazed model in order to design the corresponding loss function according to the image difference between the first dehazed image and the second dehazed image, and then realize the training of the dehazed model in the foggy branch using the foggy image.
[0044] In one possible implementation, obtaining a second synthetic foggy image based on the foggy image includes: A second depth map corresponding to the foggy image is obtained based on the depth estimation model, and based on a second predicted depth matrix corresponding to the second depth map, the foggy image is fogged using a target prior estimation algorithm and an atmospheric scattering model to obtain a second synthetic foggy image.
[0045] In the embodiments of this application, Figure 2 As shown, the depth estimation model for the foggy branch is the same as that for the non-fog branch. However, during the fogging process, a target prior estimation algorithm is added to estimate the foggy image and obtain the corresponding atmospheric light. This is to preserve the atmospheric light characteristics of the foggy image as much as possible during the fogging process. The target prior estimation algorithm used in this embodiment is the Dark Channel Prior (DCP) algorithm. This algorithm analyzes the fog model of the image based on the prior assumption that the intensity of certain color channels is often very low in local non-sky areas, and then uses the dark channel prior to accurately obtain the atmospheric light.
[0046] In one possible implementation, fogging is performed on a foggy image using a target prior estimation algorithm and an atmospheric scattering model, including: Based on the second predicted depth matrix, a second atmospheric light matrix corresponding to the second predicted depth matrix is obtained using a target prior estimation algorithm; Establishing a second atmospheric scattering coefficient numerical interval, randomly selecting any value from the second atmospheric scattering coefficient numerical interval, and constructing a second atmospheric scattering coefficient matrix corresponding to the second predicted depth matrix; Based on the second predicted depth matrix, the second atmospheric scattering coefficient matrix and the second atmospheric light matrix, the foggy image is fogged using the atmospheric scattering model.
[0047] In the embodiments of this application, Figure 2 As shown, the dehazing model is trained in the foggy branch to learn the characteristic differences between the foggy image and the second synthetic foggy image that has been fogged again. This allows the model to simulate the dynamic changes in real-world haze, thereby improving the dehazing model's ability to effectively remove haze of varying concentrations. Because the second synthetic foggy image, obtained using a randomly generated second atmospheric light matrix, has little correlation with the characteristics of the foggy image and exhibits significant feature deviation, making it difficult to train, a fixed second atmospheric light matrix is generated using a target prior estimation algorithm, rather than one generated using random depth values.
[0048] S104, establishing a total loss function based on the image difference between each first dehazed image and the second dehazed image corresponding to the first dehazed image; In the embodiments of this application, Figure 2 As shown in the figure The first loss function corresponding to the fog-free branch is used to train unpaired fog-free images to optimize the parameters of the dehazing model. The total loss function is established to train foggy and fog-free images simultaneously, so that the dehazing network can learn the relationship between the fog-free, foggy, and synthetic foggy images.
[0049] In one possible implementation, establishing a total loss function based on the image difference between each first dehazed image and a second dehazed image corresponding to the first dehazed image includes: Establishing a third loss function based on image feature deviations between each first dehazed image and the second dehazed image corresponding to the first dehazed image; Establishing a depth loss function based on the image depth deviations of each foggy image, the first dehazed image corresponding to the foggy image, and the second dehazed image; A semantic loss function is established based on the image semantic deviations of each foggy image, the first dehazed image corresponding to the foggy image, and the second dehazed image, and a second loss function is established based on the semantic loss function, the depth loss function, and the third loss function; A total loss function is established based on the first loss function and the second loss function.
[0050] In the embodiments of this application, Figure 2 As shown, represents the second loss function, represents the third loss function, represents the depth loss function, represents the semantic loss function.
[0051] During the training process of the dehazing network, the image consistency correlation features, semantic correlation features and image depth correlation features between the foggy image and the second synthetic foggy image are learned through the third loss function, semantic loss function and depth loss function respectively, thereby improving the model generalization ability when processing different concentrations of haze.
[0052] In one embodiment, the second loss function is implemented by the following formula: ; ; ; ; in, represents the second loss function, represents the third loss function, represents the depth loss function, represents the semantic loss function, represents the depth balance parameter, represents the semantic balance parameter, represents the first dehazed image, represents the second dehazed image, Indicates a foggy image. represents the L1 loss function, represents the depth estimation operation, represents the semantic extraction operation, Indicates KL divergence calculation.
[0053] In the embodiments of this application, and are all constants, and their values can be set according to actual needs. The total loss function can be obtained by summing or weighted summing the first loss function and the second loss function. In this embodiment, the process of establishing the total loss function is shown in the formula: ,in represents the total loss function.
[0054] S105 , performing loss calculations on each fog-free image and the first synthetic foggy image, and each foggy image and the second synthetic foggy image based on the total loss function, and training the pre-trained defogging model based on the loss calculation results.
[0055] In one possible implementation, based on the total loss function, loss calculation is performed on each fog-free image and the first synthetic foggy image, and each foggy image and the second synthetic foggy image, respectively, and a pre-trained defogging model is trained based on the loss calculation results, including: The first synthetic foggy image corresponding to each fog-free image is input into the first pre-trained model, and the second synthetic foggy image corresponding to each foggy image and the foggy image is synchronously input into two pre-trained defogging models respectively; Calculate a first loss of the first synthetic foggy image corresponding to the fog-free image and the fog-free image based on the first loss function, and calculate a second loss of the second synthetic foggy image corresponding to the foggy image and the foggy image based on the second loss function, The pre-trained dehazing model is trained based on the calculation results of the first loss and the second damage.
[0056] In the embodiments of this application, Figure 2 As shown, before the defogging model is trained through the total loss function, it has been trained through a large number of first loss functions and fog-free images and first synthetic foggy images to obtain a pre-trained defogging model. Therefore, it is necessary to input each fog-free image and the corresponding first synthetic foggy image, and each foggy image and the corresponding second synthetic foggy image into two sets of input channels of the defogging model for synchronous training, that is, the training of the foggy branch and the fog-free branch is synchronous, and the parameters of the pre-trained defogging model are synchronously updated based on the training results. This synchronous training method can avoid separate training, such as training the fog-free branch first and then the foggy branch, which causes the haze features learned by the defogging model to be greatly biased, and it is impossible to effectively learn the haze feature relationship between the fog-free image, the foggy image and the synthetic foggy image that is fogged again. Therefore, the method for training a defogging model based on unpaired images provided in this embodiment can effectively learn haze features close to the real scene, and effectively improve the defogging effect of the defogging model.
[0057] like Figure 4 As shown, Figure 4 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include: a processor (Processor) 401, a communication interface (Communications Interface) 402, a memory (Memory) 403, and a communication bus 404. The processor 401, the communication interface 402, and the memory 403 communicate with each other via the communication bus 404. The processor 401 can call software instructions in the memory 403 to execute the methods described in the above embodiments.
[0058] In addition, the logic instructions in the aforementioned memory 403 can be implemented in the form of a software functional unit and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application.
[0059] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.
[0060] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.
[0061] It is understood that the processor in the embodiments of the present application may be a CPU (Central Processing Unit), other general-purpose processors, DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.
[0062] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, ROM (Read-only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.
[0063] The above embodiments can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. A computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. Available media can include magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0064] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.
[0065] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.
Claims
1. A method for training a defogging model based on unpaired images, characterized in that: include: Obtain multiple foggy images and multiple fog-free images; Obtaining a first synthetic foggy image based on the fog-free image, and training a defogging model based on each of the fog-free images and the first synthetic foggy image corresponding to the fog-free image to obtain a pre-trained defogging model; Obtaining a second synthetic foggy image based on the foggy image, and inputting the foggy image and the second synthetic foggy image into the pre-trained defogging model respectively to obtain a first defogged image corresponding to the foggy image and a second defogged image corresponding to the second synthetic foggy image; establishing a total loss function based on an image difference between each of the first dehazed images and the second dehazed image corresponding to the first dehazed image; Based on the total loss function, loss calculation is performed on each of the fog-free images and the first synthetic foggy image, and each of the foggy images and the second synthetic foggy image, and the pre-trained defogging model is trained based on the results of the loss calculation.
2. The method for training a defogging model based on unpaired images according to claim 1, characterized in that: The obtaining of a first synthetic foggy image based on the fog-free image includes: A first depth map corresponding to the fog-free image is obtained based on a depth estimation model, and based on a first predicted depth matrix corresponding to the first depth map, the fog-free image is fogged using an atmospheric scattering model to obtain a first synthetic foggy image.
3. The method for training a defogging model based on unpaired images according to claim 2, characterized in that: The step of performing fogging processing on the fog-free image by using an atmospheric scattering model includes: Establishing a first atmospheric light value interval, randomly selecting any value from the first atmospheric scattering coefficient value interval, and constructing a first atmospheric light matrix corresponding to the first predicted depth matrix; Establishing a first atmospheric scattering coefficient numerical interval, randomly selecting any value from the first atmospheric scattering coefficient numerical interval, and constructing a first atmospheric scattering coefficient matrix corresponding to the first predicted depth matrix; Based on the first predicted depth matrix, the first atmospheric scattering coefficient matrix and the first atmospheric light matrix, fogging is performed on the fog-free image using an atmospheric scattering model.
4. The method for training a defogging model based on unpaired images according to claim 1, characterized in that: The training of the defogging model based on each of the fog-free images and the first synthetic foggy image corresponding to the fog-free image includes: Based on each of the fog-free images and the first synthetic foggy image corresponding to the fog-free image, a first loss function is established, and the loss of the defogging result image of the fog-free image and the first synthetic foggy image corresponding to the fog-free image is calculated based on the first loss function, and the parameters of the defogging model are trained and updated based on the loss calculation result.
5. The method for training a defogging model based on unpaired images according to claim 4, characterized in that: The first loss function is implemented by the following formula: ， in, represents the first loss function, It represents the defogging result image of the first synthetic foggy image, represents a fog-free image, represents the L1 loss function.
6. The method for training a defogging model based on unpaired images according to claim 1, characterized in that: The obtaining a second synthetic foggy image based on the foggy image includes: A second depth map corresponding to the foggy image is obtained based on a depth estimation model, and based on a second predicted depth matrix corresponding to the second depth map, the foggy image is fogged using a target prior estimation algorithm and an atmospheric scattering model to obtain a second synthetic foggy image.
7. The method for training a defogging model based on unpaired images according to claim 6, characterized in that: The fogging process of the foggy image using a target prior estimation algorithm and an atmospheric scattering model includes: Based on the second predicted depth matrix, obtaining a second atmospheric light matrix corresponding to the second predicted depth matrix using a target prior estimation algorithm; Establishing a second atmospheric scattering coefficient numerical interval, randomly selecting any value from the second atmospheric scattering coefficient numerical interval, and constructing a second atmospheric scattering coefficient matrix corresponding to the second predicted depth matrix; Based on the second predicted depth matrix, the second atmospheric scattering coefficient matrix and the second atmospheric light matrix, fogging is performed on the foggy image using an atmospheric scattering model.
8. The method for training a defogging model based on unpaired images according to claim 4, wherein: The establishing of a total loss function based on an image difference between each of the first dehazed images and the second dehazed images corresponding to the first dehazed images includes: establishing a third loss function based on image feature deviations between each of the first dehazed images and the second dehazed image corresponding to the first dehazed image; Establishing a depth loss function based on the image depth deviations of the foggy images, the first dehazed images corresponding to the foggy images, and the second dehazed images; Establishing a semantic loss function based on the image semantic deviations of each of the foggy images, the first dehazed image corresponding to the foggy image, and the second dehazed image, and establishing a second loss function based on the semantic loss function, the depth loss function, and the third loss function; A total loss function is established based on the first loss function and the second loss function.
9. The method for training a defogging model based on unpaired images according to claim 8, characterized in that: The second loss function is implemented by the following formula: ； ； ； ； in, represents the second loss function, represents the third loss function, represents the depth loss function, represents the semantic loss function, represents the depth balance parameter, represents the semantic balance parameter, represents the first dehazed image, represents the second dehazed image, Indicates a foggy image. represents the L1 loss function, represents the depth estimation operation, represents the semantic extraction operation, Indicates KL divergence calculation.
10. The method for training a defogging model based on unpaired images according to claim 8, characterized in that: The method further comprises: performing loss calculation on each of the fog-free images and the first synthetic foggy image, and each of the foggy images and the second synthetic foggy image based on the total loss function, and training the pre-trained defogging model based on the loss calculation results, including: Inputting the first synthetic foggy image corresponding to each of the fog-free images into a first pre-trained model, and inputting each of the foggy images and the second synthetic foggy image corresponding to the foggy image into the two pre-trained defogging models simultaneously; Calculating a first loss between the fog-free image and the first synthetic foggy image corresponding to the fog-free image based on the first loss function, and calculating a second loss between the foggy image and the second synthetic foggy image corresponding to the foggy image based on the second loss function, The pre-trained dehazing model is trained based on calculation results of the first loss and the second damage.