Denoising diffusion implicit model inversion method, electronic equipment and storage medium
By weighting the addition of latent variables within a specific time step interval, the inversion process of the implicit model of denoising diffusion is optimized, and the problems of large errors and calculation overhead in the prior art are solved, and an efficient inversion effect is achieved.
Patent Information
- Application Number
- CN202411075016.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2025-01-03
AI Technical Summary
The inversion method of the existing denoising diffusion implicit model has great errors and calculation overhead, especially in the iterative optimization process, which leads to inefficient in the inversion process.
In a specific time step interval, the weighted latent variables of the previous time step are weighted and added to the latent variables of the current time step, and the inversion process is optimized by calculating the formula, and the inversion is used outside the specific time step interval. Avoid additional computational overhead.
The inversion quality is improved, the calculation overhead is reduced, and only some memory consumption is needed is increased, which realizes the inversion process of the implicit model of denoising and diffusion.
Smart Images
Figure CN119251077A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the downstream task of the text-to-image generation task in deep learning, and can apply text-based graph technology to the technical field of real image editing, and particularly refers to an inversion method, electronic device and storage medium of a denoising diffusion implicit model. Background Art
[0002] In recent years, denoising diffusion implicit models have developed rapidly and have been applied in fields such as animation and visual arts. Such models can generate high-quality images based on a series of prompt words input by the user. Compared with manual painting, their speed is greatly improved while the quality is not inferior. Therefore, the idea of using denoising diffusion implicit models for real image editing came into being. Generally speaking, such models start with an initial noise, and gradually denoise the noise under the guidance of the prompt words to generate the final image result. Then, to use it for real image editing, the first thing to do is to find the initial noise corresponding to any real image for any denoising diffusion implicit model. After this initial noise is denoised by the denoising diffusion implicit model, the image result obtained is the real image, and the process of finding the initial noise corresponding to the real image is called the inversion of the denoising diffusion implicit model.
[0003] The problem faced by the existing technology is that the earliest inversion method is to directly use the denoising noise of the model as the added noise, which inevitably results in large errors; later improved methods are mostly through iterative optimization, repeatedly generating added noise, and gradually optimizing it in the process, but this will also cause the computational overhead of the inversion process to become very large, because each step of noise generation requires calling the denoising diffusion implicit model. Summary of the invention
[0004] The purpose of the present invention is to provide an inversion method, electronic device and storage medium for a denoising diffusion implicit model, so as to solve the problems existing in the prior art and optimize the inversion process of the denoising diffusion implicit model without any training and without increasing the computational overhead.
[0005] In order to achieve the above object, the solution of the present invention is: A de-noising diffusion implicit model inversion method, in the total number of de-noising steps Within, execute separately according to different time step intervals: In a specific time step interval, the weighted latent variable of the previous time step is weighted and added to the latent variable of the current time step to obtain the weighted latent variable, and its calculation formula is expressed as: ; The meaning of the specific time step interval is: for models with more than 2.0B model parameters, the time step interval is ; For models with less than 2.0B model parameters, in the time step interval ;, , Respectively represent Time step, The weighted latent variables at the time step, Latent variables at time steps The input image is encoded by the model's built-in encoder; Indicates The latent variables of the time step are calculated based on the existing baseline method DDIM Inversion; represents the weighting parameter, which is set to 0.85; Outside of a specific time step interval, only DDIM Inversion is used. Reverse.
[0006] An electronic device comprises a processor, a memory and an application program; the application program is stored in the memory and is configured to execute the inversion method of the denoising diffusion implicit model by the processor.
[0007] A computer-readable storage medium stores a computer program; when the computer program is executed in a computer, the computer is caused to execute the inversion method of the denoising diffusion implicit model.
[0008] After adopting the above technical solution, the present invention has the following technical effects: The present invention mainly improves the inversion quality by weighted addition of the latent variables of the time step and the latent variables of the previous time step within a specific time step interval, and there is no additional computational overhead, only a slight increase in memory consumption is required, thereby achieving the optimization of the inversion process of the denoising diffusion implicit model in a way that does not increase the computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 This is a graph showing image results generated according to a specific embodiment of the present invention. DETAILED DESCRIPTION
[0010] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.
[0011] The present invention discloses a method for reversing a denoising diffusion implicit model, the specific steps of which are: Within, execute separately according to different time step intervals: In a specific time step interval (this interval can be adjusted according to different models), the weighted latent variable of the previous time step is weighted and added to the latent variable of the current time step to obtain the weighted latent variable, and its calculation formula is expressed as: ; The meaning of the specific time step interval is: for models with more than 2.0B model parameters, the time step interval is ; For models with less than 2.0B model parameters, in the time step interval ;, , Respectively represent Time step, The weighted latent variables at the time step, Latent variables at time steps (i.e., the initial latent variable) is obtained by encoding the input image using the model’s own encoder; Indicates The latent variables of the time step are calculated based on the existing baseline method DDIM Inversion; represents the weighting parameter, usually set to 0.85; Outside of a specific time step interval, only DDIM Inversion is used. Reverse.
[0012] Through the above scheme, the present invention mainly improves the inversion quality by weighted addition of the latent variable of the time step and the latent variable of the previous time step within a specific time step interval, and there is no additional calculation overhead, only a slight increase in memory consumption is required. The final generated image result can be seen in Figure 1 .
[0013] The present invention also discloses an electronic device, comprising a processor, a memory and an application program; the application program is stored in the memory and is configured so that the processor executes the above-mentioned inversion method.
[0014] The present invention also discloses a computer-readable storage medium, which stores a computer program; when the computer program is executed in a computer, the computer executes the above-mentioned reversal method. The storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0015] The following are some experimental data of the present invention.
[0016] (1) See the following table, which is based on the SD-V1-4 model, and quantitatively analyzes the present invention and some existing technologies (such as DDIMInversion, ReNoise, and Fixed-Point-Iteration). The results are as follows:
[0017] It can be seen that the method steps of the present invention have better editing results than some existing well-known methods.
[0018] (2) See the following table for experimental data of quantitative analysis in full-precision mode and half-precision mode:
[0019] It can be seen that the present invention has even better performance in half-precision mode (generally speaking, the model efficiency is higher but the performance is reduced in half-precision mode), which shows the robustness of the present invention and its very suitable for scenarios that require model efficiency.
[0020] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.
Claims
1. A method for inverting a denoising diffusion implicit model, characterized in that In the total number of denoising steps Within, execute separately according to different time step intervals: In a specific time step interval, the weighted latent variable of the previous time step is weighted and added to the latent variable of the current time step to obtain the weighted latent variable, and its calculation formula is expressed as: ； The meaning of the specific time step interval is: for models with more than 2.0B model parameters, the time step interval is ; For models with less than 2.0B model parameters, in the time step interval ;, , Respectively represent Time step, The weighted latent variables at the time step, Latent variables at time steps The input image is encoded by the model's built-in encoder; Indicates The latent variables of the time step are calculated based on the existing baseline method DDIM Inversion; represents the weighting parameter, which is set to 0.85; Outside of a specific time step interval, only DDIM Inversion is used. Reverse.
2. An electronic device, characterized in that: The invention comprises a processor, a memory and an application program; the application program is stored in the memory and is configured to execute the inversion method of the denoising diffusion implicit model as claimed in claim 1 by the processor.
3. A computer-readable storage medium, characterized in that: The storage medium stores a computer program; when the computer program is executed in a computer, the computer is caused to execute the inversion method of the denoising diffusion implicit model according to claim 1.