A text generation image method and device

A technology for generating images and texts, applied in the field of image generation methods and devices based on text descriptions, to achieve the effects of complete content, reduced training difficulty, and reasonable logic

Active Publication Date: 2019-03-29
南京德磐信息科技有限公司
View PDF7 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] The technical problem to be solved by the present invention is to aim at the deficiencies in the above-mentioned prior art, and provide a text description based on conditional generation confront

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A text generation image method and device
  • A text generation image method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The present invention will be further illustrated below in conjunction with the accompanying drawings and specific embodiments, and it should be understood that these examples are only used to illustrate the present invention and are not intended to limit the scope of the present invention. After reading the present invention, modifications to various equivalent forms of the present invention by those skilled in the art fall within the scope defined by the appended claims of the present application.

[0035] A text-to-image method based on conditional generative adversarial networks and recurrent neural networks, such as figure 1 shown, including the following steps:

[0036] Step 1, build a text encoder, input a natural language text sequence, and output an embedded representation of the text. The natural language text sequence is a word sequence p = (w 1 ,w 2 ,...,w d ), where each word is represented by a pre-trained word vector.

[0037] For example: input the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a text generation image method and device. The text generation image method comprises the following steps: step 1, coding a natural language text describing the image to obtaina text semantic embedded representation; 2, mixing the text semantic embedding representation obtained in the step 1 with random noise, reading the text semantic embedding representation by a cyclicneural network transcoder, outputting the object hidden coding of each step by the random noise and the hidden state of the cyclic neural network transcoder; 3, decoding the hidden coding of each stepobject output from the step 2 to generate step images, and finally fusing all the step images to obtain the generated images; 4, carrying out confrontation training on that generate image and the real image. The generator of the invention decodes and generates the foreground and background pixel set of an image according to the object hidden coding through multi-step transcoding, and fuses the foreground and background pixel set of the image to generate a high-quality image, thereby reducing the training difficulty of directly generating the image.

Description

technical field [0001] The invention relates to the technical field of deep learning generation models, in particular to an image generation method and device based on text description. Background technique [0002] Generating photorealistic images from natural language text descriptions is an important problem and has wide applications, such as photo editing, computer-aided design, etc. [0003] There are many ways to learn generative models across image and text modalities. One of these lines of research is learning generative models for text conditioned on images, called "image captioning". The current mainstream processing flow of this type of model is to first extract image features and encode them with an encoder, and then use a decoder to generate unstructured text. [0004] Recently, Generative Adversarial Network (GAN) has achieved good results in generating realistic images. Its variant conditional-GAN (referred to as cGAN) can generate images related to the mea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/22G06T9/00
CPCG06T9/002G06F40/126
Inventor 周德宇胡名起蒋明敏
Owner 南京德磐信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap