A text generation image method and device

Abstract

The invention discloses a text generation image method and device. The text generation image method comprises the following steps: step 1, coding a natural language text describing the image to obtaina text semantic embedded representation; 2, mixing the text semantic embedding representation obtained in the step 1 with random noise, reading the text semantic embedding representation by a cyclicneural network transcoder, outputting the object hidden coding of each step by the random noise and the hidden state of the cyclic neural network transcoder; 3, decoding the hidden coding of each stepobject output from the step 2 to generate step images, and finally fusing all the step images to obtain the generated images; 4, carrying out confrontation training on that generate image and the real image. The generator of the invention decodes and generates the foreground and background pixel set of an image according to the object hidden coding through multi-step transcoding, and fuses the foreground and background pixel set of the image to generate a high-quality image, thereby reducing the training difficulty of directly generating the image.

Description

technical field [0001] The invention relates to the technical field of deep learning generation models, in particular to an image generation method and device based on text description. Background technique [0002] Generating photorealistic images from natural language text descriptions is an important problem and has wide applications, such as photo editing, computer-aided design, etc. [0003] There are many ways to learn generative models across image and text modalities. One of these lines of research is learning generative models for text conditioned on images, called "image captioning". The current mainstream processing flow of this type of model is to first extract image features and encode them with an encoder, and then use a decoder to generate unstructured text. [0004] Recently, Generative Adversarial Network (GAN) has achieved good results in generating realistic images. Its variant conditional-GAN (referred to as cGAN) can generate images related to the mea...

AI Technical Summary

Problems solved by technology

[0009] The technical problem to be solved by the present invention is to aim at the deficiencies in the above-mentioned prior art, and provide a text description based on conditional generation confrontation network and cyclic neural network A method for generating images, which can solve the problem of image generation in the case of multiple entities and complex relationships in the text description

Images