Method for generating image by sensing combined space attention text

Abstract

The invention discloses a method for generating an image by combining perception and spatial attention texts. The method comprises the following steps: generating an initial image, and drawing the basic shape and color of an object; image refining is carried out, and new image features refined from the spatial dimension and the word importance dimension are generated; the refining process comprises the following steps: fusing fine-grained word-level text information and image information, eliminating defects and adding details to an initial image in combination with a space attention mechanism and a dynamic memory mechanism, and enhancing image feature region characterization; by training a target function, a generator is encouraged to generate an image which is more real and more in line with text semantics. The method ensures that the generator focuses on the content, the position and the shape of the image sub-region corresponding to the most relevant word, avoids the randomness in the generation process, reduces the difference between the finally generated image and the target image, can improve the perception quality and the layout of the text generation image, and improves the text generation efficiency. The image can be efficiently and accurately generated.

Description

technical field [0001] The invention belongs to the field of text synthesis images, and in particular relates to a method for generating images of texts with combined perception and spatial attention. Background technique [0002] Text-to-image synthesis techniques hold great promise for applications in areas such as art generation and computer-aided design. Now, using text to generate images can not only greatly reduce the cost of matching images for text creators, but also improve the efficiency of computer creation. Therefore, it is highly necessary to find more efficient ways to generate realistic high-resolution images for text-image synthesis. [0003] Generative Adversarial Network (GAN for short) is based on the idea of ​​game theory, and constructs a generator model and a discriminator model through a deep neural network. The generator generates samples with random noise as input, and the discriminator judges whether the generated samples are real or not. In the ...

AI Technical Summary

Problems solved by technology

For example, when the object category is not described in the text, the content of the generated image may be very different from the real image

Furthermore, although multi-stage methods are by far the best generative methods, when dealing with complex text with many objects (such as the COCO dataset), they cannot focus and refine all objects well, so the quality of the generated results is not accurate

Images