A low-light image degradation simulation method and system based on conditional offset diffusion trajectory

Abstract

The present application relates to the technical field of image processing and generation, and particularly relates to a low-light image degradation simulation method and system based on conditional offset diffusion trajectory, comprising the following steps: S1, obtaining a diffusion model trained on a clear image dataset, and using a deterministic reverse sampling process thereof to define a reference trajectory from noise to a clear image; S2, constructing a conditional offset network, taking a trajectory state, a time step and a degradation condition vector as input, outputting a disturbance vector, and completing training under frozen diffusion model parameters; S3, inputting a clear image, generating an initial state through forward noise addition, iteratively sampling along the reference trajectory and superimposing the disturbance, and finally outputting a low-light image meeting the degradation condition through a decoder. The present application introduces a conditional disturbance mechanism to offset the reference trajectory of the diffusion model, realizes efficient and controllable modeling of the low-light image degradation process, and has the advantages of stable training, accurate degradation, high-quality generated data and the like.

Description

Technical Field

[0001] This invention relates to the field of image processing and generation technology, and in particular to a method and system for simulating low-light image degradation based on conditional offset diffusion trajectory. Background Technology

[0002] In low-light image processing tasks, high-quality degraded image samples are crucial for training models for image enhancement, denoising, and recognition. However, due to complex acquisition conditions and the scarcity of real-world paired data, constructing large-scale, structured datasets with diverse degradation characteristics remains a significant challenge. With the powerful capabilities of diffusion models in image generation, their potential reversible modeling properties offer a new approach to building high-fidelity, controllable image degradation simulation mechanisms.

[0003] Existing image degradation methods mainly rely on manual rules (such as random dimming and noise addition) or training end-to-end degradation generation networks, which have certain limitations. On the one hand, manual enhancement methods cannot simulate the multi-scale and nonlinear characteristics of the real degradation process, and the generated images lack structural consistency. On the other hand, directly training degradation generation models not only involves a large number of parameters and high convergence difficulty, but also heavily relies on large-scale real-world paired data, which can easily lead to problems such as mode collapse and unnatural degradation, making it difficult to meet the needs of real-world low-light image modeling. Summary of the Invention

[0004] This invention provides a method and system for simulating low-light image degradation based on conditional migration diffusion trajectories. By introducing the generated trajectory of a pre-trained diffusion model as a priori path, a lightweight conditional migration network is constructed to perform controllable perturbations on each state, thereby achieving fine-tuning of degradation type and intensity. This not only improves training stability and simulation accuracy, but also generates a large number of high-quality degradation images that conform to the expected distribution, providing rich and reliable data support for image processing tasks in low-light scenes.

[0005] A method for simulating low-light image degradation based on conditional migration diffusion trajectories includes the following steps: S1, Obtain the diffusion model on the clear image dataset, and use its deterministic backsampling process to define the baseline trajectory from random noise to clear image; S2, a conditional migration network is constructed based on the diffusion model. The current trajectory state, time step information and degradation condition vector are taken as inputs, and the output is a perturbation vector. Under the condition of freezing the diffusion model parameters, the conditional migration network is optimized by using paired data composed of clear images and their corresponding real low-light images. S3. For any input sharp image, an initial hidden state is generated through the forward noise addition process of the diffusion model. The initial hidden state is used as the starting point of the degradation trajectory. Starting from the maximum time step, the process is iteratively executed along the reference trajectory. The conditional offset network is called to generate the perturbation vector of the current step, which is superimposed on the next state predicted by the diffusion model to form a degradation trajectory that deviates from the sharp path. This process continues until all time steps are sampled. Finally, the decoder of the diffusion model restores the final hidden state to the low-light image corresponding to the degradation condition vector.

[0006] Optionally, S1 includes: S11, Train a diffusion model on a large-scale, clear image dataset. The diffusion model includes a forward noise addition process and a backward noise reduction process. The training objective is to learn a noise prediction network. ; S12, choosing the denoising diffusion implicit model, based on a fixed noise scheduler and noise prediction network, defines the noise latent variables from random noise. To clear image The baseline trajectory of the mapping.

[0007] Optionally, S2 includes: S21, Designing a Lightweight Neural Network As a conditional offset network, its input components include the current trajectory state. (representing the diffusion trajectory) Image latent variables of each step), time step information and the degeneracy condition vector and output the perturbation vector. ; S22, embed the conditional offset network into the deterministic sampling path of the diffusion model, and predict the baseline state at each step. Applying a perturbation generates a new degenerate trajectory. ; S23, using paired data That is, using real sharp images and corresponding real low-light images as supervision signals to optimize the parameters of the conditional offset network. This makes the final generated image approximate the real degraded image.

[0008] Optionally, S3 includes: S31, for any input sharp image By utilizing the forward noise addition process in the diffusion model, it is transformed into a process that occurs at the maximum time step. The initial hidden state , as the starting point of the degradation trajectory; S32, from time step Initially, sampling is performed iteratively according to the preset baseline trajectory structure, at each time step. The baseline predicted state is calculated using a frozen diffusion model. , the current state Time step With the user-specified degradation condition vector Input the conditional offset network to obtain the perturbation vector. Then, by superimposing perturbations, a degraded trajectory state that deviates from the clear image generation path is formed, which is used as the input for the next iteration until sampling is completed; S33, when iterating to time step At that time, the final hidden state is obtained. Subsequently, the initial hidden state is restored to a visible image in the image space through the decoder module in the diffusion model, generating a vector that meets the specified degradation conditions. The low-light image required by the control is used as the final output.

[0009] Optionally, S31 includes: S311, Determine the maximum time step of the diffusion model. And calculate the first noise dispatcher based on the selected noise dispatcher. cumulative noise figure of the step ; S312, Generate a sampled Gaussian noise with the same dimensions as the image to perform a noise addition operation; S313 uses cumulative noise figure and sampled Gaussian noise to create a clear image. Convert to the first The initial hidden state of the step .

[0010] Optionally, S32 includes: S321, from the maximum time step Start by iterating gradually towards time step 0, at each time step The frozen diffusion model is based on the current state. Calculate the corresponding baseline prediction state That is, the trajectory points followed by which a clear image is generated without disturbance; S322, change the current state Time step With the user-specified degradation condition vector These are also used as input to the conditional offset network to generate the perturbation vector for the current time step; S323, apply the perturbation vector to the baseline prediction state to obtain the degraded trajectory state of the current step, which is used as the input for the next step, and repeat the perturbation application process according to the time steps from... Until Iterate step by step until the final hidden state is obtained. .

[0011] Optionally, S33 includes: S331, when the conditional perturbation iterative sampling process reaches time step... When the iteration stops, the currently obtained trajectory state is taken as the final hidden state of the degenerate trajectory. ; S332, final hidden state Decoder module corresponding to the input diffusion model This is used to map the image back from the latent space to obtain a visual image. ; S333, decode the obtained visual image As the final output, the visual image corresponds to the user-specified degradation condition vector in terms of visual characteristics. Correspondingly, it exhibits brightness distribution, noise level, and color shift effects that conform to the target's low light degradation characteristics.

[0012] A low-light image degradation simulation system based on conditional migration diffusion trajectory, used to implement the aforementioned low-light image degradation simulation method based on conditional migration diffusion trajectory, includes the following modules: Diffusion Model Module: A diffusion model is pre-trained on a sharp image dataset, and a baseline trajectory from random noise to a sharp image is defined based on the deterministic backsampling process of the diffusion model. Conditional offset network module: Based on the current trajectory state, time step information and degradation condition vector, a perturbation vector is generated, and the network is trained and optimized using paired data consisting of clear images and corresponding real low-light images under the condition of freezing the parameters of the diffusion model module; Degraded image generation module: Converts any input clear image into an initial hidden state through the forward noise addition process of the diffusion model module, and uses the hidden state as the starting point of the degradation trajectory. Iteratively performs sampling along the reference trajectory, generates perturbation at each time step and superimposes it onto the diffusion model prediction state to form a degradation trajectory that deviates from the clear path. Decoding output module: After the iteration is completed, the final hidden state is input into the decoder of the diffusion model module to generate a low-light image corresponding to the degradation condition vector.

[0013] The beneficial effects of this invention are: This invention introduces the deterministic backsampling trajectory of a pre-trained diffusion model as a baseline path and designs a conditional offset network to directionally perturb this path. This avoids the computational complexity and stability problems caused by directly training a complete degradation model. Only a lightweight neural network needs to be optimized to generate high-quality degradation images, effectively improving training efficiency and system stability.

[0014] This invention achieves continuous and controllable adjustment of image degradation features in the diffusion latent space by embedding a user-defined multidimensional degradation condition vector into the perturbation generation at each time step in the sampling process. This includes brightness reduction, noise enhancement, and color shift, resulting in a generation result with good diversity and interpretability, meeting the simulation needs of different low-light scenes.

[0015] This invention, by fusing clear image priors and conditional perturbation information from a diffusion model during the degradation image generation process, can generate synthetic samples that are highly consistent with the distribution of real low-light images, constructing a structured training dataset that does not require manual annotation, and providing high-quality and scalable data support for downstream tasks such as low-light image enhancement, denoising, and recognition. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the simulation method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the system functional modules according to an embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Those skilled in the art may employ other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0019] like Figure 1 As shown, a low-light image degradation simulation method based on conditional migration diffusion trajectory includes the following steps: S1, Obtain the diffusion model on the clear image dataset, and use its deterministic backsampling process to define the baseline trajectory from random noise to clear image; S2, a conditional migration network is constructed based on the diffusion model. The current trajectory state, time step information and degradation condition vector are taken as inputs, and the output is a perturbation vector. Under the condition of freezing the diffusion model parameters, the conditional migration network is optimized by using paired data composed of clear images and their corresponding real low-light images. S3. For any input sharp image, an initial hidden state is generated through the forward noise addition process of the diffusion model. The initial hidden state is used as the starting point of the degradation trajectory. Starting from the maximum time step, the process is iteratively executed along the baseline trajectory. The conditional offset network is called to generate the perturbation vector of the current step, which is superimposed on the next state predicted by the diffusion model to form a degradation trajectory that deviates from the sharp path. This process continues until all time steps are sampled. Finally, the decoder of the diffusion model restores the final hidden state to the low-light image corresponding to the degradation condition vector.

[0020] S1 includes: S11. Train a diffusion model on a large-scale, clear image dataset. The diffusion model includes a forward noise addition process and a backward noise reduction process. The training objective is to learn a noise prediction network. The training loss function is expressed as: ; in, For a clear image, Standard Gaussian noise, For the first The image after adding noise. This is the cumulative noise figure. For sampling time steps; S12, choosing the denoising diffusion implicit model, based on a fixed noise scheduler and noise prediction network, defines the noise latent variables from random noise. To clear image The baseline trajectory of the mapping is represented as: ; ; in, This refers to the state at the previous time step. A clear image based on the current state estimate.

[0021] S2 includes: S21, Designing a Lightweight Neural Network As a conditional offset network, its input components include the current trajectory state. (representing the diffusion trajectory) Image latent variables of each step), time step information and the degeneracy condition vector and output the perturbation vector. , is represented as: ; ; in, For time step information The embedding vector representation, For joint input vectors; S22, embed the conditional offset network into the deterministic sampling path of the diffusion model, and predict the baseline state at each step. Applying a perturbation generates a new degenerate trajectory. , is represented as: ; in, This is the state after the disturbance. The baseline predicted state is given by the frozen diffusion model. This is the disturbance intensity coefficient; S23, using paired data That is, using real sharp images and corresponding real low-light images as supervision signals to optimize the parameters of the conditional offset network. To make the final generated image approximate the real degraded image, the optimization objective function is expressed as: ; in, For the final degraded image, The difference metric function is used to freeze all parameters of the diffusion model during training, optimizing only the differences. .

[0022] S3 includes: S31, for any input sharp image By utilizing the forward noise addition process in the diffusion model, it is transformed into a process that occurs at the maximum time step. The initial hidden state , as the starting point of the degradation trajectory; S32, from time step Initially, sampling is performed iteratively according to the preset baseline trajectory structure, at each time step. The baseline predicted state is calculated using a frozen diffusion model. , the current state Time step With the user-specified degradation condition vector Input the conditional offset network to obtain the perturbation vector. Then, by superimposing perturbations, a degraded trajectory state that deviates from the clear image generation path is formed, which is used as the input for the next iteration until sampling is completed; S33, when iterating to time step At that time, the final hidden state is obtained. Subsequently, the initial hidden state is restored to a visible image in the image space through the decoder module in the diffusion model, generating a vector that meets the specified degradation conditions. The low-light image required by the control is used as the final output.

[0023] S31 includes: S311, Determine the maximum time step of the diffusion model. And calculate the first noise dispatcher based on the selected noise dispatcher. cumulative noise figure of the step This is used to control the proportion of information retained in a sharp image, and is represented as: ; ; in, The retention coefficient for each step, For the first The noise intensity of the step; S312 generates a sampled Gaussian noise with the same dimensions as the image to perform a noise addition operation, represented as: ; in, To sample Gaussian noise; S313 uses cumulative noise figure and sampled Gaussian noise to create a clear image. Convert to the first The initial hidden state of the step , is represented as: .

[0024] S32 includes: S321, from the maximum time step Start by iterating gradually towards time step 0, at each time step The frozen diffusion model is based on the current state. Calculate the corresponding baseline prediction state The trajectory points that generate a clear image without disturbance are represented as: ; S322, change the current state Time step With the user-specified degradation condition vector These are also used as input to the conditional offset network to generate the perturbation vector for the current time step; S323, apply the perturbation vector to the baseline prediction state to obtain the degraded trajectory state of the current step, which is used as the input for the next step, and repeat the perturbation application process according to the time steps from... Until Iterate step by step until the final hidden state is obtained. .

[0025] S33 includes: S331, when the conditional perturbation iterative sampling process reaches time step... When the iteration stops, the currently obtained trajectory state is taken as the final hidden state of the degenerate trajectory. , is represented as: ; in, In time step Below, the degenerate trajectory state after perturbation; S332, final hidden state Decoder module corresponding to the input diffusion model This is used to map the image back from the latent space to obtain a visual image. , is represented as: ; S333, decode the obtained visual image As the final output, the visual image corresponds to the user-specified degradation condition vector in terms of visual characteristics. Correspondingly, it exhibits brightness distribution, noise level, and color shift effects that conform to the target's low light degradation characteristics.

[0026] like Figure 2 As shown, a low-light image degradation simulation system based on conditional offset diffusion trajectory is used to implement the aforementioned low-light image degradation simulation method based on conditional offset diffusion trajectory, and includes the following modules: Diffusion Model Module: Pre-trains a diffusion model on a sharp image dataset and defines a baseline trajectory from random noise to a sharp image based on the deterministic backsampling process of the diffusion model; Conditional offset network module: Based on the current trajectory state, time step information and degradation condition vector, a perturbation vector is generated. Under the condition of freezing the parameters of the diffusion model module, the network is trained and optimized using paired data consisting of clear images and corresponding real low-light images. Degraded Image Generation Module: Converts any input sharp image into an initial hidden state through the forward noise addition process of the diffusion model module, and uses this hidden state as the starting point of the degradation trajectory. Iteratively performs sampling along the baseline trajectory, generates perturbation at each time step and superimposes it onto the diffusion model prediction state to form a degradation trajectory that deviates from the sharp path. Decoding output module: After the iteration is completed, the final hidden state is input into the decoder of the diffusion model module to generate a low-light image corresponding to the degradation condition vector.

[0027] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0028] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for simulating low-light image degradation based on conditional migration diffusion trajectories, characterized in that, Includes the following steps: S1, Obtain the diffusion model on the clear image dataset, and use its deterministic backsampling process to define the baseline trajectory from random noise to clear image; S2, a conditional migration network is constructed based on the diffusion model. The current trajectory state, time step information and degradation condition vector are taken as inputs, and the output is a perturbation vector. Under the condition of freezing the diffusion model parameters, the conditional migration network is optimized by using paired data composed of clear images and their corresponding real low-light images. S3. For any input sharp image, an initial hidden state is generated through the forward noise addition process of the diffusion model. The initial hidden state is used as the starting point of the degradation trajectory. Starting from the maximum time step, the process is iteratively executed along the reference trajectory. The conditional offset network is called to generate the perturbation vector of the current step, which is superimposed on the next state predicted by the diffusion model to form a degradation trajectory that deviates from the sharp path. This process continues until all time steps are sampled. Finally, the decoder of the diffusion model restores the final hidden state to the low-light image corresponding to the degradation condition vector.

2. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 1, characterized in that, S1 includes: S11, Train a diffusion model on a large-scale, clear image dataset. The diffusion model includes a forward noise addition process and a backward noise reduction process. The training objective is to learn a noise prediction network. ; S12, choosing the denoising diffusion implicit model, based on a fixed noise scheduler and noise prediction network, defines the noise latent variables from random noise. To clear image The baseline trajectory of the mapping.

3. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 2, characterized in that, S2 includes: S21, Designing a Lightweight Neural Network As a conditional offset network, its input components include the current trajectory state. Time step information and the degeneracy condition vector and output the perturbation vector. ; S22, embed the conditional offset network into the deterministic sampling path of the diffusion model, and predict the baseline state at each step. Applying a perturbation generates a new degenerate trajectory. ; S23, using paired data That is, using real sharp images and corresponding real low-light images as supervision signals to optimize the parameters of the conditional offset network. This makes the final generated image approximate the real degraded image.

4. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 3, characterized in that, S3 includes: S31, for any input sharp image By utilizing the forward noise addition process in the diffusion model, it is transformed into a process that occurs at the maximum time step. The initial hidden state , as the starting point of the degradation trajectory; S32, from time step Initially, sampling is performed iteratively according to the preset baseline trajectory structure, at each time step. The baseline predicted state is calculated using a frozen diffusion model. , the current state Time step With the user-specified degradation condition vector Input the conditional offset network to obtain the perturbation vector. Then, by superimposing perturbations, a degraded trajectory state that deviates from the clear image generation path is formed, which is used as the input for the next iteration until sampling is completed; S33, when iterating to time step At that time, the final hidden state is obtained. Subsequently, the initial hidden state is restored to a visible image in the image space through the decoder module in the diffusion model, generating a vector that meets the specified degradation conditions. The low-light image required by the control is used as the final output.

5. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 4, characterized in that, S31 includes: S311, Determine the maximum time step of the diffusion model. And calculate the first noise dispatcher based on the selected noise dispatcher. cumulative noise figure of the step ; S312, Generate a sampled Gaussian noise with the same dimensions as the image to perform a noise addition operation; S313 uses cumulative noise figure and sampled Gaussian noise to create a clear image. Convert to the first The initial hidden state of the step .

6. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 5, characterized in that, S32 includes: S321, from the maximum time step Start by iterating gradually towards time step 0, at each time step The frozen diffusion model is based on the current state. Calculate the corresponding baseline prediction state That is, the trajectory points followed by which a clear image is generated without disturbance; S322, change the current state Time step With the user-specified degradation condition vector These are also used as input to the conditional offset network to generate the perturbation vector for the current time step; S323, apply the perturbation vector to the baseline prediction state to obtain the degraded trajectory state of the current step, which is used as the input for the next step, and repeat the perturbation application process according to the time steps from... Until Iterate step by step until the final hidden state is obtained. .

7. The low-light image degradation simulation method based on conditional offset diffusion trajectory according to claim 6, characterized in that, S33 includes: S331, when the conditional perturbation iterative sampling process reaches time step... When the iteration stops, the currently obtained trajectory state is taken as the final hidden state of the degenerate trajectory. ; S332, final hidden state Decoder module corresponding to the input diffusion model This is used to map the image back from the latent space to obtain a visual image. ; S333, decode the obtained visual image As the final output, the visual image corresponds to the user-specified degradation condition vector in terms of visual characteristics. Correspondingly, it exhibits brightness distribution, noise level, and color shift effects that conform to the target's low light degradation characteristics.

8. A low-light image degradation simulation system based on conditional offset diffusion trajectory, used to implement the low-light image degradation simulation method based on conditional offset diffusion trajectory as described in any one of claims 1-7, characterized in that, Includes the following modules: Diffusion Model Module: A diffusion model is pre-trained on a sharp image dataset, and a baseline trajectory from random noise to a sharp image is defined based on the deterministic backsampling process of the diffusion model. Conditional offset network module: Based on the current trajectory state, time step information and degradation condition vector, a perturbation vector is generated, and the network is trained and optimized using paired data consisting of clear images and corresponding real low-light images under the condition of freezing the parameters of the diffusion model module; Degraded image generation module: Converts any input clear image into an initial hidden state through the forward noise addition process of the diffusion model module, and uses the hidden state as the starting point of the degradation trajectory. Iteratively performs sampling along the reference trajectory, generates perturbation at each time step and superimposes it onto the diffusion model prediction state to form a degradation trajectory that deviates from the clear path. Decoding output module: After the iteration is completed, the final hidden state is input into the decoder of the diffusion model module to generate a low-light image corresponding to the degradation condition vector.

Images