Image defogging method and device combining feedback control and diffusion model, equipment and medium

Abstract

The invention discloses an image defogging method and device combining feedback control and a diffusion model, equipment and a medium, and the method comprises the steps: taking the simulation of the difference between the features of a defogged image and a fogless image as a loss target, and carrying out the combined optimization of a preset image defogging backbone network and a first foggy prior extraction module; based on a one-dimensional vector, the denoising network and the second foggy prior extraction module are assisted to carry out iterative optimization, and the second foggy prior extraction module and the first foggy prior extraction module are the same in structure; and inputting the optimized modulation feature and the fog image into the preset image defogging backbone network, and iteratively optimizing a defogging result of the fog image through frequency domain comparison and a double-feedback channel based on a feedback control theory to obtain a target simulation defogging image corresponding to the fog image. The invention belongs to the field of image defogging. According to the invention, the image defogging accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image dehazing, and more particularly to an image dehazing method, apparatus, device, and medium that combines feedback control and a diffusion model. Background Technology

[0002] In the field of image dehazing technology, traditional methods are mostly based on physical models (such as atmospheric scattering models) for parameter estimation and image restoration. However, these methods are easily limited by model assumptions and are prone to distortion and color deviation in complex or dense fog scenes. In recent years, deep learning methods have significantly improved dehazing performance by learning the mapping relationship between foggy and fog-free images through end-to-end networks.

[0003] However, existing deep dehazing networks generally suffer from strong dependence on input priors and insufficient generalization ability, especially when dealing with foggy images outside the training set distribution, resulting in unstable recovery performance. Furthermore, most methods are feedforward single-inference methods, lacking a dynamic adjustment mechanism for the dehazing results, making it difficult to finely restore high-frequency details and realistic textures. Therefore, to more accurately dehaze foggy images and restore fog-free images as much as possible, this invention provides an image dehazing method that combines feedback control and a diffusion model. Summary of the Invention

[0004] This invention provides an image dehazing method, apparatus, device, and medium that combines feedback control and a diffusion model. This solves the technical problem of low image restoration when dehazing foggy images in the prior art, and achieves the technical effect of improving the accuracy of image dehazing.

[0005] In a first aspect, the present invention provides an image dehazing method that combines feedback control and a diffusion model, comprising: a first training phase and a second training phase; The first training phase, S11, includes S110-S112: S110, input both the fog image and the fog-free image into the first fog-precedent extraction module to obtain a one-dimensional vector. and ; S111, combining fog map and one-dimensional vector and They are input together into the preset image dehazing backbone network to obtain the simulated dehazing image corresponding to the hazy image; S112, taking the difference in features between simulated dehazed images and haze-free images as the loss target, jointly optimize the preset image dehazing backbone network and the first hazy prior extraction module; The second training phase, S12, includes S120-S121: S120, for a one-dimensional vector and Perform forward diffusion noise addition to obtain as well as ; S121, input the fog map into the second fog prior extraction module to obtain a one-dimensional vector. as well as And based on one-dimensional vectors , The auxiliary denoising network and the second foggy prior extraction module are iteratively optimized, wherein the second foggy prior extraction module has the same structure as the first foggy prior extraction module; Fog image, one-dimensional vector and The optimized second fog prior extraction module is input to obtain the corresponding optimized modulation features. The optimized modulation features and fog map are then input into the preset image defogging backbone network. Based on feedback control theory, the defogging result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated defogging image corresponding to the fog map.

[0006] Furthermore, regarding one-dimensional vectors and ,include: Transform the atmospheric scattering model:

[0007] in, This is the initial fog map. For global atmospheric light, For the transmission diagram, The image is a fog-free image obtained after transforming the original fog image using an atmospheric scattering model. Simplified to feature space, including:

[0008] in, For global atmospheric light in the feature space, This is a transmission map in the characteristic space. The feature map of the fog-free image. The feature map of the initial fog map; Further simplification, including:

[0009] in, , ; After inputting both the fog image and the fog-free image into the first fog-prior extraction module, ,in For the first fog-aware prior extraction module For channel splicing, Fog image, This is a fog-free image.

[0010] Furthermore, for a one-dimensional vector and Perform forward diffusion noise addition to obtain as well as ,include:

[0011] in, It follows a normal distribution. This is the noisy prior before adding noise. For the first Noisy priors after step diffusion, Diffusion coefficient The cumulative product, It is the identity matrix. It is a forward conditional probability distribution.

[0012] Furthermore, it also includes: After forward diffusion noise addition, denoising is performed, including:

[0013] Noise prediction is performed based on the second foggy prior extraction module and the denoising network, including:

[0014] in, For the first The prior after denoising, For the first The diffusion coefficient of the step, For noise, For the first The prior after denoising, It is an inverse conditional probability distribution. The mean of a Gaussian distribution is given. Let be the variance of the Gaussian distribution.

[0015] Furthermore, the fog map is input into the second fog prior extraction module to obtain a one-dimensional vector. as well as ,include: Use by This yields a one-dimensional vector. as well as ,in, This is the second fog-a priori extraction module.

[0016] Furthermore, fog maps and one-dimensional vectors... and The optimized second foggy prior extraction module is input to obtain the corresponding optimized modulation features, including: The fog map is input into the optimized second fog prior extraction module, which outputs a one-dimensional vector. and ; a one-dimensional vector and Input the encoder of the preset image dehazing backbone network to modulate the features of the hazy image and obtain the modulated features; The modulation features are subjected to multi-scale acquisition, frequency domain processing, and spatial domain processing to obtain the corresponding optimized modulation features.

[0017] Furthermore, based on feedback control theory, the dehazing result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated dehazed image corresponding to the fog map, including: The highest-level features output by the encoder are input into the decoder of the preset image dehazing backbone network to obtain the first dehazing result; The frequency domain spectrum of the fog map is introduced, and the frequency domain comparison is performed on the first defogging result; Comparative features are obtained based on internal feedback and main feedback. The comparison features are iteratively optimized to obtain the target simulated dehazed image corresponding to the fog map.

[0018] Secondly, the present invention provides an image dehazing device combining feedback control and a diffusion model, comprising: The first training module is used to execute the first training phase S11, including S110-S112: S110, the fog image and the fog-free image are input together into the first fog-precedence extraction module to obtain a one-dimensional vector. and S111, combining fog map and one-dimensional vector and The images are input together into the preset image dehazing backbone network to obtain the simulated dehazing image corresponding to the fog image; S112, taking the difference in features between the simulated dehazing image and the fog-free image as the loss target, the preset image dehazing backbone network and the first fog-precedence extraction module are jointly optimized. The first training module is used to execute the second training phase S12, including S120-S121: S120, for one-dimensional vectors... and Perform forward diffusion noise addition to obtain as well as S121, Input the fog map into the second fog prior extraction module to obtain a one-dimensional vector. as well as And based on one-dimensional vectors , The auxiliary denoising network and the second foggy prior extraction module are iteratively optimized, wherein the second foggy prior extraction module has the same structure as the first foggy prior extraction module; The iterative dehazing module is used to dehaze fog maps and one-dimensional vectors. and The optimized second fog prior extraction module is input to obtain the corresponding optimized modulation features. The optimized modulation features and fog map are then input into the preset image defogging backbone network. Based on feedback control theory, the defogging result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated defogging image corresponding to the fog map.

[0019] Thirdly, the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute an image dehazing method that combines feedback control and a diffusion model, as provided in the first aspect.

[0020] Fourthly, the present invention provides a non-transitory computer-readable storage medium, wherein when the instructions in the non-transitory computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to execute an image dehazing method that combines feedback control and a diffusion model as provided in the first aspect.

[0021] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention proposes an image dehazing method combining feedback control and a diffusion model, achieving synergistic optimization of dehazing performance through two-stage training. The first stage extracts prior information from both hazy and hazy images and supervises the dehazing network to generate a preliminary dehazed image. The second stage introduces a diffusion model to learn noise addition and denoising of the prior, enhancing its robustness, and outputs modulation features through a second hazy prior extraction module. Finally, a feedback control mechanism is introduced into the dehazing backbone network, iteratively optimizing the dehazing results through frequency domain analysis and dual feedback channels, improving the detail and realism of image restoration, and achieving a superior dehazing effect.

[0022] This invention constructs an image dehazing method, FCIDM, that combines an atmospheric scattering model and a diffusion model and incorporates feedback control. This invention effectively achieves image dehazing by proposing the combination of atmospheric scattering and diffusion models at the feature level, ensuring the physical interpretability of feature processing in the image feature space. Simultaneously, it incorporates the processing of high and low frequency information in both the spatial and frequency domains during feature processing. The proposed introduction of feedback control and progressive dehazing effectively enhances the performance of image dehazing tasks. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic flowchart of an image dehazing method combining feedback control and a diffusion model provided by the present invention; Figure 2 This is a schematic diagram of the structure of the preset image dehazing backbone network provided by the present invention; Figure 3 This is a schematic diagram of the structure of the fog-aware prior extraction module provided by the present invention; Figure 4 This is a schematic diagram of the structure of the multi-scale spatial frequency domain attention module provided by the present invention; Figure 5 This is a schematic diagram of the structure of the dynamic fog map attention module provided by the present invention; Figure 6 This is a schematic diagram of the spatial frequency domain attention module provided by the present invention; Figure 7 A schematic diagram of the closed-loop control structure and the corresponding decoder provided by the present invention; Figure 8 This is a schematic diagram of the frequency domain comparator provided by the present invention; Figure 9 A schematic diagram of the forward diffusion and reverse denoising structure provided by the present invention; Figure 10 This is a schematic diagram of the structure for fusing the decoder and encoder provided by the present invention. Detailed Implementation

[0025] This invention provides an image dehazing method that combines feedback control and a diffusion model, thereby solving the technical problem of low image restoration accuracy when dehazing foggy images in the prior art.

[0026] The technical solution of this invention is to solve the above-mentioned technical problems, and the overall idea is as follows: An image dehazing method combining feedback control and a diffusion model includes: a first training phase and a second training phase; the first training phase S11 includes S110-S112: S110, the fogged image and the fog-free image are input together into a first foggy prior extraction module to obtain a one-dimensional vector. and S111, combining fog map and one-dimensional vector and The two images are input together into a preset image dehazing backbone network to obtain a simulated dehazed image corresponding to the hazy image; S112, using the difference in features between the simulated dehazed image and the hazy-free image as the loss target, the preset image dehazing backbone network and the first hazy prior extraction module are jointly optimized; the second training phase S12 includes S120-S121: S120, for one-dimensional vectors and Perform forward diffusion noise addition to obtain as well as S121, Input the fog map into the second fog prior extraction module to obtain a one-dimensional vector. as well as And based on one-dimensional vectors , The auxiliary denoising network and the second fog prior extraction module are iteratively optimized, wherein the second fog prior extraction module has the same structure as the first fog prior extraction module; the fog map and one-dimensional vector are then used for the optimization. and The optimized second fog prior extraction module is input to obtain the corresponding optimized modulation features. The optimized modulation features and fog map are then input into the preset image defogging backbone network. Based on feedback control theory, the defogging result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated defogging image corresponding to the fog map.

[0027] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0028] First, it should be clarified that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0029] This invention modulates image features with physical meaning based on an atmospheric scattering model introduced at the feature level and performs progressive dehazing of the image using feedback control in the frequency domain. The network generates physical priors at the feature scale based on the atmospheric scattering model using a diffusion model, and performs progressive dehazing of the image by introducing feedback control in the frequency domain. The network structure includes high- and low-frequency feature processing modules in both the spatial and frequency domains, enhancing the performance of the image dehazing task through a two-stage supervised training approach. Please refer to the following for details: This invention provides, for example Figure 1 An image dehazing method combining feedback control and diffusion model includes: a first training phase S11, a second training phase S12, and S13.

[0030] S11, the first training phase S11, includes S110-S112: S110, input both the fog image and the fog-free image into the first fog-precedent extraction module to obtain a one-dimensional vector. and .

[0031] Regarding one-dimensional vectors and ,include: Estimating fog priors in the feature space not only maintains theoretical interpretability but also avoids using the transmission map and true values ​​of atmospheric light. This involves transforming the atmospheric scattering model:

[0032] in, This is the initial fog map. For global atmospheric light, For the transmission diagram, The image is a fog-free image obtained after transforming the original fog image using an atmospheric scattering model. Simplified to feature space, including:

[0033] in, For global atmospheric light in the feature space, This is a transmission map in the characteristic space. The feature map of the fog-free image. The feature map of the initial fog map; Further simplification, including:

[0034] in, , ; After inputting both the fog image and the fog-free image into the first fog-prior extraction module, ,in For the first fog-aware prior extraction module For channel splicing, Fog image, This is a fog-free image.

[0035] like Figure 3 As shown in the first training phase, fog map and fog-free images They will be input together into the first foggy prior extraction module, that is: .

[0036] The fog image and the fog-free image are stitched together, input into the fog prior extraction module, and then output. (Fog a priori).

[0037] S111, combining fog map and one-dimensional vector and They are input together into the preset image dehazing backbone network ( Figure 2 ), to obtain the simulated dehazed image corresponding to the fog image.

[0038] Extract The image features are fed into the encoder of the preset image dehazing backbone network to perform physical meaning modulation on the hazy image features, specifically including: The backbone network encoder first extracts the feature map of the fog map; then it uses the output of the first fog prior extraction module. Input encoder, according to For the initial feature map M Element-level modulation is performed to obtain preliminary optimized features.

[0039] S112, taking the difference in features between the simulated dehazed image and the haze-free image as the loss target, jointly optimizes the preset image dehazing backbone network and the first hazy prior extraction module.

[0040] Using the difference in features between simulated dehazed images and haze-free images as the loss objective, the preset image dehazing backbone network and the first haze prior extraction module are jointly optimized, and the neural network parameters of the preset image dehazing backbone network and the first haze prior extraction module are updated synchronously. Minimize the loss of features between the dehazed and hazy images, ensure that the features extracted by the first hazy prior extraction module can be accurately modulated, and at the same time, the preset image dehazing backbone network can effectively utilize the prior output of high-quality dehazing-related features. Through multiple iterations, the first foggy prior extraction module can output more reliable priors based on paired data, and the preset image dehazing backbone network can better utilize the priors to complete feature optimization, thus completing the first training stage.

[0041] The second training phase, S12, includes S120-S121. Since the neural network's input for dehazing is only the fog image, a prior extractor with the same structure as the HPEM from the first phase but different input can be trained in this second phase. Similar to knowledge distillation, the HPEM with only the fog image input needs to use a diffusion model to learn the prior knowledge learned in the first phase. Diffusion noise addition and orientation denoising can be found in [reference needed]. Figure 9 : It should be noted that the illustrated process only includes one-dimensional vectors. Noise addition and denoising are performed, but for one-dimensional vectors... Adding and removing noise and one-dimensional vectors The similarities are not detailed here, therefore, they will not be repeated in this invention.

[0042] S120, for a one-dimensional vector and Perform forward diffusion noise addition to obtain as well as ,include:

[0043] in, It follows a normal distribution. The noisy prior before adding noise (i.e., the foggy prior output by the first foggy prior extraction module) ), For the first Noisy priors after step diffusion, Diffusion coefficient The cumulative product, It is the identity matrix. It is a forward conditional probability distribution.

[0044] Also includes: After forward diffusion noise addition, denoising is performed, including:

[0045] Noise prediction is performed based on the second foggy prior extraction module and the denoising network, including:

[0046] in, For the first The prior after denoising, For the first The diffusion coefficient of the step, For noise, For the first The prior after denoising, It is an inverse conditional probability distribution. The mean of a Gaussian distribution is given. Let be the variance of the Gaussian distribution.

[0047] S121, input the fog map into the second fog prior extraction module to obtain a one-dimensional vector. as well as And based on one-dimensional vectors , The auxiliary denoising network and the second fog prior extraction module are iteratively optimized, wherein the second fog prior extraction module has the same structure as the first fog prior extraction module (i.e., Figure 3 ).

[0048] Use by This yields a one-dimensional vector. as well as ,in, This is the second fog-a priori extraction module.

[0049] The fog image is input separately into the second fog prior extraction module, which extracts the features of the fog image and outputs a unique conditional vector (one-dimensional). as well as , correspond , corresponding , as well as Locate features in the fog map that are relevant to the prior information to prevent the denoising process from deviating from the actual situation of the fog map.

[0050] The conditional vector Dη is input into the denoising network to help it predict noise more accurately. The optimization objective of the denoising network is then to combine... After noise reduction As close as possible to the output of the second foggy prior extraction module Meanwhile, the loss of the optimization objective will be backpropagated to the second foggy prior extraction module, adjusting the neural network parameters of the second foggy prior extraction module so that the output of the second foggy prior extraction module can more effectively assist in noise reduction.

[0051] pass Define the probability distribution for reverse denoising to ensure that the denoised prior is consistent with the prior distribution of the second foggy prior extraction module, and complete knowledge distillation.

[0052] S13, combining fog map and one-dimensional vector and The optimized second fog prior extraction module is input to obtain the corresponding optimized modulation features. The optimized modulation features and fog map are then input into the preset image defogging backbone network. Based on feedback control theory, the defogging result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated defogging image corresponding to the fog map.

[0053] Fog image, one-dimensional vector and The optimized second foggy prior extraction module is input into the fog image to obtain the corresponding optimized modulation features, including: inputting the fog image into the optimized second foggy prior extraction module and outputting a one-dimensional vector. and ; Transform a one-dimensional vector and Input the encoder of the preset image dehazing backbone network to modulate the features of the hazy image to obtain modulated features; perform multi-scale capture, frequency domain processing and spatial domain processing on the modulated features to obtain the corresponding optimized modulated features.

[0054] To capture feature information at different scales, a multi-scale spatial frequency domain attention module was designed. Figure 4 The multi-scale spatial frequency domain attention module captures features at different scales through average pooling operations.

[0055] The spatial frequency domain feature extraction module is used to process features at each scale in both the spatial and frequency domains.

[0056] In the frequency domain, the dynamic fog map attention module uses filters generated from the phase and amplitude spectra of the fog map to dynamically filter the phase and amplitude spectra of the feature map.

[0057] Taking the phase spectrum as an example, such as Figure 5 As shown, for the input fog map I and feature map M, the phase spectrum is first separated, and then the phase spectrum of the fog map is used to generate a filter:

[0058] in, It is an average pooling operation. It is batch regularization. It is a parameter matrix used to generate dynamic filters for low-frequency or high-frequency blurred image amplitudes. , It is the hyperbolic tangent function. The phase spectrum is then dynamically filtered and modulated, and finally, the modulated amplitude spectrum and phase spectrum are converted to the spatial domain.

[0059] For spatial domain information, the spatial-frequency domain attention module (such as...) Figure 6 (As shown) The feature map is divided into high-frequency and low-frequency components for processing. A stripe feature map is used to approximate global attention at low cost, and a 3×3 convolution is used to extract local attention.

[0060] Taking local attention extraction as an example, convolution can be performed on the feature map to obtain low-frequency features. The high-frequency components of the feature map can be obtained through... The high-frequency and low-frequency information is then input into the feature modulation module.

[0061] High-frequency and low-frequency information are concatenated along the channel dimension, then global average pooling and global max pooling are applied, followed by convolutional and sigmoid layers to obtain two weight matrices. The weight matrix is ​​multiplied by either the high-frequency or low-frequency component of the feature map, and then summed to obtain the adaptively modulated feature result. Finally, by weighting the stripe feature map and the square feature map, the output feature map in the spatial domain can be obtained.

[0062] Based on feedback control theory, the dehazing result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated dehazed image corresponding to the fog map, including: The highest-level features output by the encoder are input into the decoder of the preset image dehazing backbone network to obtain the first dehazing result; The frequency domain spectrum of the fog map is introduced, and the frequency domain comparison is performed on the first defogging result; Comparative features are obtained based on internal feedback and main feedback. The comparison features are iteratively optimized to obtain the target simulated dehazed image corresponding to the fog map.

[0063] For image dehazing, the features output from the encoder are used as the input to the first decoder to obtain the first dehazing result without comparison steps.

[0064] According to feedback control theory, the comparison process requires a clean image as a pre-input. However, in the inference process of deep learning, a clean image cannot be introduced into the network like a fogged image; therefore, the initial dehazing result cannot be directly compared with the clean image.

[0065] However, after comparing the amplitude and phase spectra of the initial dehazing results with those of the fog image, the distance between the amplitude and phase spectra of the initial dehazing and those of the clean image is greater in most regions compared to the distance between the amplitude and phase spectra of the fog image and those of the clean image. In other words, the fog image is closer to the target clean image in the frequency domain than the initial dehazing result in most regions.

[0066] Therefore, the amplitude spectrum and phase spectrum of the fog image can be introduced as approximate preset target values. However, unlike the feedback control system which directly subtracts the preset target value from the initial processing result, a gating mechanism is designed to adaptively learn which parts of the image in the frequency domain have poor recovery effects after the initial defogging.

[0067] exist Figure 7 In this design, the decoding block acts as the forward transfer function G of the control system. Furthermore, two feedback channels are designed: one is the transfer function H1 of the convolutional layer as the inner feedback path, and the other is the transfer function H2 of the unity function as the main feedback path.

[0068] Assume the initial input is , It is the output feature of the i-th coded block. This is the nth defogging result. The first defogging result needs to be... The results and fog map are input into a frequency domain comparator (FCM). Figure 8 ).

[0069] Applying Fourier transform to the input fog map and the first dehazing result yields the result obtained from... and The amplitude and phase spectra of the fog map are shown. and The amplitude and phase spectra of the first defogging result are shown. Since the phase and amplitude spectra of the fog map do not contain all the clean information.

[0070] Therefore, the classical comparison operation of subtraction in control theory cannot be used directly.

[0071] An adaptive gating structure is used to determine the region of poor recovery in the phase spectrum and amplitude spectrum of the first defogging result, thereby achieving a comparison operation similar to feedback control.

[0072] Then, by Generated features and encoder features The results are fed back to the comparison module via an internal feedback channel to modulate the initial comparison results.

[0073] in This represents the feature of passing through G and H1 for the j-th time. Figure 9 The fusion operation in the text represents fusion. and The process, This is a feature fusion operation.

[0074] To avoid information forgetting in deep learning, features can be introduced from the encoder block. In order to obtain the final comparison features.

[0075] Taking into account both model parameters and performance, the total number of iterations is set to three. The formula for the nth dehazing iteration is as follows:

[0076]

[0077] in, For the feedback comparison module, This is the result of the nth defogging attempt. For defogging generator.

[0078] However, the information in the fog map is coupled. If a decoder with shared parameters is used for stepwise defogging, the effect may not be ideal. Considering that multi-scale defogging helps the network learn different levels of fog information, the three-level feedback defogging structure is changed to a structure without shared parameters, and the output size of each defogging module is specified to be different, thus obtaining the final defogging decoder.

[0079] In summary, this invention proposes an image dehazing method combining feedback control and a diffusion model, achieving synergistic optimization of dehazing performance through two-stage training. The first stage utilizes both hazy and hazy images to extract prior information and supervises the dehazing network to generate a preliminary dehazed image. The second stage introduces a diffusion model to learn noise addition and denoising of the prior information, enhancing its robustness, and outputs modulation features through a second hazy prior extraction module. Finally, a feedback control mechanism is introduced into the dehazing backbone network, using frequency domain analysis and dual feedback channels to iteratively optimize the dehazing results, improving the detail and realism of image restoration and achieving superior dehazing performance.

[0080] This invention constructs an image dehazing method, FCIDM, that combines an atmospheric scattering model and a diffusion model and incorporates feedback control. This invention effectively achieves image dehazing by proposing the combination of atmospheric scattering and diffusion models at the feature level, ensuring the physical interpretability of feature processing in the image feature space. Simultaneously, it incorporates the processing of high and low frequency information in both the spatial and frequency domains during feature processing. The proposed introduction of feedback control and progressive dehazing effectively enhances the performance of image dehazing tasks.

[0081] Based on the same inventive concept, this invention provides an image dehazing device that combines feedback control and a diffusion model, comprising: The first training module is used to execute the first training phase S11, including S110-S112: S110, the fog image and the fog-free image are input together into the first fog-precedence extraction module to obtain a one-dimensional vector. and S111, combining fog map and one-dimensional vector and The images are input together into the preset image dehazing backbone network to obtain the simulated dehazing image corresponding to the fog image; S112, taking the difference in features between the simulated dehazing image and the fog-free image as the loss target, the preset image dehazing backbone network and the first fog-precedence extraction module are jointly optimized. The first training module is used to execute the second training phase S12, including S120-S121: S120, for one-dimensional vectors... and Perform forward diffusion noise addition to obtain as well as S121, Input the fog map into the second fog prior extraction module to obtain a one-dimensional vector. as well as And based on one-dimensional vectors , The auxiliary denoising network and the second foggy prior extraction module are iteratively optimized, wherein the second foggy prior extraction module has the same structure as the first foggy prior extraction module; The iterative dehazing module is used to dehaze fog maps and one-dimensional vectors. and The optimized second fog prior extraction module is input to obtain the corresponding optimized modulation features. The optimized modulation features and fog map are then input into the preset image defogging backbone network. Based on feedback control theory, the defogging result of the fog map is iteratively optimized through frequency domain comparison and dual feedback channels to obtain the target simulated defogging image corresponding to the fog map.

[0082] Based on the same inventive concept, the present invention also provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute an image dehazing method that combines feedback control and a diffusion model, as described above.

[0083] Based on the same inventive concept, the present invention also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute an image dehazing method that combines feedback control and a diffusion model as described above.

[0084] Since the electronic device described in this embodiment is an electronic device used to implement the information processing method in the embodiments of the present invention, those skilled in the art can understand the specific implementation methods and various variations of the electronic device in this embodiment based on the information processing method described in the embodiments of the present invention. Therefore, how the electronic device implements the method in the embodiments of the present invention will not be described in detail here. Any electronic device used by those skilled in the art to implement the information processing method in the embodiments of the present invention falls within the scope of protection of the present invention.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0090] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An image defogging method combining feedback control with a diffusion model, characterized by, Comprising: a first training stage and a second training stage; The first training stage S11 comprises S110-S112: S110, input the foggy image and the non-foggy image into the first foggy prior extraction module to obtain a one-dimensional vector and ; S111, input the fog image, one-dimensional vector and to a preset image defogging backbone network to obtain a simulated defogging image corresponding to the fog image. S112, taking the difference between the simulated defogging image and the feature of the haze-free image as the loss target, jointly optimizing the preset image defogging backbone network and the first haze prior extraction module; The second training stage S12 comprises S120-S121: S120, performing forward diffusion and noise adding on the one-dimensional vector and to obtain and ; S121, input the fog map into the second foggy prior extraction module to obtain a one-dimensional vector and based on the one-dimensional vector , aid the iterative optimization of the denoising network and the second foggy prior extraction module, wherein the second foggy prior extraction module has the same structure as the first foggy prior extraction module; the fog map, a one-dimensional vector and The second foggy prior extraction module is input into the optimized second foggy prior extraction module, and the corresponding optimized modulation feature is obtained. The optimized modulation feature and the fog map are input into the preset image defogging backbone network, and based on the feedback control theory, the defogging result of the fog map is iteratively optimized through the frequency domain comparison and the double feedback channel, and the target simulated defogging image corresponding to the fog map is obtained. 2.The image defogging method of claim 1, wherein, With respect to one-dimensional vectors and comprise: Transform the atmospheric scattering model: wherein, is an initial fog map, is a global atmospheric light, is a transmission map, is a fog-free image obtained by transforming the original fog map through an atmospheric scattering model; Simplify to feature space, comprising: wherein, is the global atmospheric light of the feature space, is the transmission map of the feature space, is the feature map of the haze-free image, is the feature map of the initial haze image; Further simplify, comprising: wherein , ; The foggy image and the fog-free image are input into the first foggy prior extraction module, wherein the first foggy prior extraction module is a channel concatenation, is a foggy image, is a fog-free image. 3.The image defogging method combining feedback control with diffusion model according to claim 1, wherein, on a one-dimensional vector and forward diffusion plus noise, resulting in and comprising: wherein, is a normal distribution, is a noisy prior before adding noise, is a noisy prior after the first step of diffusion, is a diffusion coefficient cumulative product, is an identity matrix, is a forward conditional probability distribution. 4.The image defogging method combining feedback control with diffusion model according to claim 3, wherein, Also comprising: After completing the forward diffusion noise, denoising, comprising: Based on the second haze prior extraction module and the denoising network, noise prediction, comprising: wherein, is the first de-noised prior of step is the first diffusion coefficient of step is the noise, is the first de-noised prior of step is the reverse conditional probability distribution, is the mean of the Gaussian distribution, is the variance of the Gaussian distribution. 5.The image defogging method combining feedback control with diffusion model according to claim 4, wherein, The fog map is input into a second foggy prior extraction module to obtain a one-dimensional vector and , comprising: using the , obtaining a one-dimensional vector and wherein, is a second foggy prior extraction module.

6. The image defogging method combining feedback control with diffusion model according to claim 1, wherein, The fog map, one-dimensional vector And Input the optimized second fog prior extraction module, get the corresponding optimized modulation features, including: Input the fog map into the second optimized fog prior extraction module, output a one-dimensional vector and ; A one-dimensional vector And An encoder of the preset image defogging backbone network is input to modulate the features of the fog image to obtain modulated features; The modulation feature is captured by multiple scales, processed in the frequency domain and the spatial domain, and the corresponding optimized modulation feature is obtained.

7. The image defogging method combining feedback control with diffusion model according to claim 1, wherein, Based on feedback control theory, through frequency domain comparison and double feedback channel, iterative optimization of the defogging result of the fog image is performed to obtain the target simulated defogging image corresponding to the fog image, comprising: The highest level feature output by the encoder is input into the decoder of the preset image defogging backbone network to obtain the first defogging result; The frequency domain spectrum of the fog image is introduced, and the frequency domain comparison is performed on the first defogging result; Based on the inner feedback and the main feedback, the comparison feature is obtained; Iterative optimization is performed on the comparison feature to obtain the target simulated defogging image corresponding to the fog image.

8. An image defogging device combining feedback control with a diffusion model, characterized by, Comprising: The first training module is configured to perform a first training stage S11, including S110-S112: S110, inputting the foggy image and the haze-free image into a first haze prior extraction module to obtain a one-dimensional vector and S111, inputting the foggy image, the one-dimensional vector and and a preset image defogging backbone network to obtain a simulated defogging image corresponding to the foggy image; and S112, jointly optimizing the preset image defogging backbone network and the first haze prior extraction module by taking the difference between the features of the simulated defogging image and the haze-free image as a loss target. The first training module is configured to perform the second training stage S12, including S120-S121: S120, performing forward diffusion noise on the one-dimensional vector and to obtain and ; S121, input the fog map into the second foggy prior extraction module to obtain a one-dimensional vector and based on the one-dimensional vector 、 aid the iterative optimization of the denoising network and the second foggy prior extraction module, wherein the second foggy prior extraction module has the same structure as the first foggy prior extraction module; An iterative defogging module is configured to input the fog map and the one-dimensional vector And The second optimized foggy prior extraction module is inputted into the optimized modulation feature, and the optimized modulation feature and the fog map are inputted into the preset image defogging backbone network. Based on the feedback control theory, the defogging result of the fog map is iteratively optimized through the frequency domain comparison and the double feedback channel, and the target simulation defogging image corresponding to the fog map is obtained.

9. An electronic device, comprising: Comprising: A processor; A memory for storing instructions executable by the processor; The processor is configured to execute to realize the image defogging method combining feedback control and diffusion model as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, comprising: When the instructions in the non-transitory computer readable storage medium are executed by the processor of the electronic device, the electronic device can execute the image defogging method combining feedback control and diffusion model as claimed in any one of claims 1 to 7.

Images