A text-guided image restoration method and system

Abstract

The invention discloses a text-guided image restoration technology method and system, belonging to the field of computer vision image restoration. It mainly includes the following steps: 1) For a set of image and guide text training set, through a coarse-to-fine image inpainting module, learn the joint representation of image information and text information and generate reasonable inpainted images. 2) For the image generated by the image inpainting module, a text reconstruction module is used to learn the semantic correlation between the inpainted image and the guiding text, and infer the mask part of the guiding text. The invention adopts the guiding text guidance model to generate semantically consistent, visually coherent and controllable images, adopts a coarse-to-fine cross-modal generation network and a new type of supervisory signal guidance model to gradually repair images, and uses a text reconstruction module to guide Based on the generated inpainted images, the model predicts the masked parts of the guide text, improving visual-text consistency.

Description

technical field [0001] The invention relates to the field of image restoration, in particular to a text-guided image restoration method and system. Background technique [0002] Image inpainting is a basic and important topic in the field of computer vision, which aims to complement the missing regions of a partially masked image and output a reasonable image. Most existing image completion methods complete missing regions by extending or borrowing information from surrounding regions, and these methods work well when missing regions are similar to their surrounding regions. However, the images generated by these methods will be unsatisfactory if there is not enough relevant information available in the surrounding area. [0003] In recent years, many artificially guided image inpainting techniques have been proposed, but most of them adopt structure-based methods, such as artificially adding boundary lines, delimiting reference areas and specifying extension directions, et...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to solve the problem of lack of diversity in image repair in the prior art, introduce text-guided technology into the field of image repair, and propose a text-guided image repair method and system, using text description to guide the model to generate consistent semantics, Visually coherent and controllable images, a coarse-to-fine cross-modal generative network and a new supervisory signal guidance model are proposed to repair images step by step, and a text reconstruction module is used to guide the model to guide text based on the generated repair images Mask parts for prediction, improving visual-text consistency

Images