Image editing method based on text guided diffusion model

Abstract

The application relates to the technical field of image editing, and discloses an image editing method based on a text-guided diffusion model, which comprises the following steps: obtaining an original image to be edited and original text prompts corresponding to the original image; aligning the original image and the original text prompts through a denoising diffusion implicit model to obtain initial noise; performing inversion optimization on the original text prompts and the initial noise to generate optimized noise; and based on target text prompts, performing sampling adjustment on the optimized noise according to a gradient guiding mechanism to obtain a target edited image. The denoising diffusion implicit model is used to align the original image with the prompt words with clear structures, and the accumulated error in the inversion is corrected, the gradient guidance is introduced in the generation stage, the sampling result is gradually adjusted, the editing effect driven by the target prompt words is realized, the model training and parameter fine-tuning are completely unnecessary, the original prompt words and the target prompt words are not dependent on the length correspondence, and more flexible and more intuitive semantic editing capability is provided.

Images