Integrated image restoration method based on dynamic trajectory diffusion model, electronic equipment and storage medium

Abstract

The invention discloses an integrated image restoration method based on a dynamic trajectory diffusion model, and the method comprises the steps: 1, in a training stage, for a time step t, representing a one-step diffusion process as # imgabs0 # # imgabs1 # 2, and employing an L1 norm to carry out the constraint of a diffusion image It and a prediction image # imgabs2 # of the time step t; 3, training the neural network through back propagation; and 4, in a reasoning stage, a corresponding diffusion residual modulator # imgabs3 # and a corresponding diffusion distribution modulator # imgabs4 # are adaptively allocated according to input picture information, a picture is reasoned and output through a diffusion model, and a corresponding reverse reasoning process is expressed as # imgabs5 #. According to the method, an optimal recovery path is adaptively selected according to different image degradation types by dynamically adjusting a diffusion track, so that the image quality is improved, and the image quality is improved. Therefore, the overall performance of image restoration is improved, the calculation overhead is effectively reduced, and the flexibility of processing different degradation types is improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and computer vision technology, and relates to image restoration technology based on deep learning, especially using a diffusion model to realize various degradation restoration tasks of images. This field covers various image enhancement and restoration technologies such as image denoising, deblurring, rain removal, and defogging, and particularly refers to an integrated image restoration method, electronic device, and storage medium based on a dynamic trajectory diffusion model. Background Art

[0002] Image restoration aims to reconstruct a clear original image from a damaged or degraded image, which is of great significance in practical applications.

[0003] Traditional image restoration methods usually design separate models for specific degradation types, such as denoising, deblurring, deraining, and dehazing, which results in the need to train multiple models for each degradation type, increasing computational and storage overhead.

[0004] In recent years, integrated image restoration technology has gradually emerged, aiming to improve the flexibility and efficiency of the method by sharing distribution information between different tasks and handling multiple image degradation problems under a unified framework. Although existing diffusion models have performed well in image generation and restoration tasks, these models usually adopt static diffusion trajectories and use the same diffusion steps for all input images, ignoring the intrinsic differences between images of different degradation types, which limits the restoration effect. Summary of the invention

[0005] The main purpose of the present invention is to provide an integrated image restoration method, electronic device and storage medium based on a dynamic trajectory diffusion model to solve the problems existing in the prior art. By dynamically adjusting the diffusion trajectory, the optimal restoration path is adaptively selected according to different image degradation types, thereby improving the overall performance of image restoration. This solves the problems of low efficiency and single function of traditional methods that can only target a single degradation type, effectively reduces the computational overhead and improves the flexibility of processing different degradation types.

[0006] In order to achieve the above object, one of the solutions of the present invention is:

[0007] An integrated image restoration method based on a dynamic trajectory diffusion model, comprising:

[0008] Step 1: During the training phase, for time step t, the one-step diffusion process is expressed as:

[0009]

[0010] Among them, I t and I t-1Represent the diffusion images at time step t and time step t-1 respectively; a t Represents the original diffusion residual step length; represents the mth diffuse residual modulator; I Res represents the residual image; β t represents the original noise step length; ∈ t-1 represents the additive noise at time t-1; δ t represents the original diffusion distribution step length; represents the nth diffusion distribution modulator; I in represents the input image;

[0011] Step 2: Use L1 norm to calculate the diffusion image I at time step t t and predicted image Constraints are expressed as:

[0012]

[0013] Where L represents the loss function used for optimization; ||·||1 represents the L1 norm;

[0014] Step 3: Back propagation training of neural network;

[0015] Step 4: In the inference phase, adaptively assign the corresponding diffuse residual modulator according to the input image information and DDM And the output image is inferred through the diffusion model, and the corresponding reverse reasoning process is expressed as:

[0016]

[0017] The second solution of the present invention is:

[0018] An electronic device, characterized by comprising a processor, a memory and an application; the application is stored in the memory and is configured so that the processor executes the integrated image restoration method based on the dynamic trajectory diffusion model.

[0019] The third solution of the present invention is:

[0020] A computer-readable storage medium stores a computer program; when the computer program is executed in the computer, the computer executes the integrated image restoration method based on the dynamic trajectory diffusion model.

[0021] After adopting the above technical solution, the present invention has the following technical effects:

[0022] In the image restoration process, fixed processing steps are no longer used. Instead, the diffusion trajectory is dynamically adjusted according to the specific degradation of each image. This technical solution introduces a diffusion residual modulator and DDM The optimal restoration path can be adaptively selected, thereby improving the quality and efficiency of image restoration, effectively reducing computational overhead and improving the flexibility of handling different degradation types. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the algorithm flow of a specific embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a visualization effect of a specific embodiment of the present invention;

[0025] Figure 3 It is a task performance diagram of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.

[0027] refer to Figure 1 As shown, the present invention discloses an integrated image restoration method based on a dynamic trajectory diffusion model (DT-Diff), which is used for integrated image restoration tasks. The core of the technical solution of the present invention is to no longer use fixed processing steps in the image restoration process, but to dynamically adjust the diffusion trajectory according to the specific degradation of each image. The technical solution introduces a diffusion residual modulator and DDM Adaptively select the optimal restoration path, thereby improving the quality and efficiency of image restoration, including:

[0028] Step 1: During the training phase, for time step t, the one-step diffusion process is expressed as:

[0029]

[0030] Among them, I t and I t-1 Represent the diffusion images at time step t and time step t-1 respectively; a t Represents the original diffusion residual step length; represents the mth diffuse residual modulator; I Res represents the residual image; β t represents the original noise step length; ∈ t-1 represents the additive noise at time t-1; δ t represents the original diffusion distribution step length; represents the nth diffusion distribution modulator; I in represents the input image;

[0031] Step 2: Use L1 norm to calculate the diffusion image I at time step t t and predicted image Constraints are expressed as:

[0032]

[0033] Where L represents the loss function used for optimization; ||·||1 represents the L1 norm;

[0034] Step 3: Back propagation training of neural network;

[0035] Step 4: In the inference phase, adaptively assign the corresponding diffuse residual modulator according to the input image information and DDM And the output image is inferred through the diffusion model, and the corresponding reverse reasoning process is expressed as:

[0036]

[0037] The technical effect of the present invention is verified by experimental tests as follows.

[0038] This experiment was conducted on five tasks: image deraining, image dehazing, image deblurring, image desnowing and image low-light enhancement. The visual effect of image contrast restoration (the face part has been mosaicked) and the contrast loss between the restored image and the true image can effectively prove the effectiveness of this method.

[0039] See also Figure 2 ,In terms of visual effects, our proposed method (the second to last column) can better restore the image details and ,texture features compared to previous methods and are closer to the real image (the last column).

[0040] See also Figure 3 ,The method we proposed can effectively improve the restoration performance on various ,tasks in the integrated image restoration task, and the overall performance is improved from 30.18 to 31.05.

[0041] The present invention also discloses an electronic device, comprising a processor, a memory and an application program; the application program is stored in the memory and is configured so that the processor executes the above method.

[0042] The present invention also discloses a computer-readable storage medium, which stores a computer program; when the computer program is executed in a computer, the computer executes the above method. The storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0043] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.

Claims

1. An integrated image restoration method based on a dynamic trajectory diffusion model, characterized in that include: Step 1: During the training phase, for time step t, the one-step diffusion process is expressed as: Among them, I t and I t-1 Represent the diffusion images at time step t and time step t-1 respectively; a t Represents the original diffusion residual step size; represents the mth diffuse residual modulator; I Res represents the residual image; β t represents the original noise step length; ∈ t-1 represents the additive noise at time t-1; δ t represents the original diffusion distribution step length; represents the nth diffusion distribution modulator; I in represents the input image; Step 2: Use L1 norm to calculate the diffusion image I at time step t t and predicted image Constraints are expressed as: Where L represents the loss function used for optimization; ||·||1 represents the L1 norm; Step 3: Back propagation training of neural network; Step 4: In the inference phase, adaptively assign the corresponding diffuse residual modulator according to the input image information and DDM And the output image is inferred through the diffusion model, and the corresponding reverse reasoning process is expressed as:

2. An electronic device, characterized in that The invention comprises a processor, a memory and an application program; the application program is stored in the memory and is configured so that the processor executes the integrated image restoration method based on the dynamic trajectory diffusion model as claimed in claim 1.

3. A computer-readable storage medium, characterized in that The storage medium stores a computer program; when the computer program is executed in the computer, the computer executes the integrated image restoration method based on the dynamic trajectory diffusion model as claimed in claim 1.

Images