Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and device for generating image from text, computer equipment and storage medium

A technology for generating images and text images, applied in computer parts, computing, neural learning methods, etc., can solve problems such as poor quality and low image pixels, improve authenticity, avoid low image pixels, and enhance supervision factors Effect

Pending Publication Date: 2022-01-21
SOUTH CHINA UNIV OF TECH +1
View PDF0 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned deficiencies in the prior art, the present invention provides a method, device, computer equipment, and storage medium for generating images from text. The method gradually improves the pixels and quality of generated images by adopting multi-level generative confrontation networks, avoiding the The problem of low pixels and poor quality of the image generated by the single generation confrontation network. At the same time, an attention mechanism is added between the cascaded generators to pay attention to the important parts of the output features, which further improves the authenticity of the generated image. Thereby improving the semantic consistency between generated images and text

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for generating image from text, computer equipment and storage medium
  • Method and device for generating image from text, computer equipment and storage medium
  • Method and device for generating image from text, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0083] Such as figure 1 As shown, the present embodiment provides a method for generating images from text, comprising the following steps:

[0084] S101. Acquire a text-image pair in a database; wherein, the text-image pair includes text and an image, and the text is a descriptive text of the image as an original text.

[0085] The image-text pair data in the network is acquired through a crawler or the like, and used as a text-image pair in the database. The text in a text-image pair is a descriptive statement of the image, and there is semantic consistency between the text and the image.

[0086] In this embodiment, the entire network includes a multi-stage generative confrontation network, an image annotation network, and a twin neural network.

[0087] S102. Input the original text into the multi-level generative adversarial network to obtain a corresponding image.

[0088] Further, step S102 includes:

[0089] (1) Before inputting the original text into the multi-lev...

Embodiment 2

[0138] Such as Figure 5 As shown, this embodiment provides a device for generating images from text, which includes a text-image pair acquisition module 501, a predicted image generation module 502, a predicted text generation module 503, a similarity calculation module 504, and a multi-level generation confrontation network training Module 505 and text generation image module 506, wherein:

[0139] The text-image pair acquisition module 501 is used to acquire the text-image pair in the database; wherein, the text-image pair includes text and images, and the text is the descriptive text of the image as the original text;

[0140] A predictive image generation module 502, configured to input the original text into a multi-stage generation confrontation network to obtain a corresponding image;

[0141] A predictive text generation module 503, configured to input the corresponding image into the trained image tagging network to generate a predictive text;

[0142] A similarity...

Embodiment 3

[0147] This embodiment provides a computer device, which can be a computer, such as Figure 6 As shown, a processor 602, a memory, an input device 603, a display 604 and a network interface 605 are connected through a system bus 601, the processor is used to provide computing and control capabilities, and the memory includes a non-volatile storage medium 606 and an internal memory 607, the non-volatile storage medium 606 stores an operating system, a computer program, and a database, the internal memory 607 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium, and the processor 602 executes the During computer program, realize the method for the text generation image of above-mentioned embodiment 1, as follows:

[0148] Obtain the text-image pair in the database; wherein, the text-image pair includes text and images, and the text is descriptive text of the image as the original text;

[0149] Inputting t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a device for generating an image from a text, computer equipment and a storage medium. The method comprises the following steps: acquiring a text image pair in a database; using a text in the text image pair as an original text; inputting the original text into a multi-stage generative adversarial network to obtain a corresponding image; inputting the corresponding image into the trained image annotation network to generate a prediction text; inputting the predicted text and the original text into the trained twin neural network to obtain the similarity between the predicted text and the original text; according to the similarity, training the multi-stage generative adversarial network to obtain a trained multi-stage generative adversarial network; and inputting a text input by a user into the trained multi-stage generative adversarial network to generate an image corresponding to the text. According to the invention, the multi-stage generative adversarial network is adopted, the pixel and quality of the generated image are gradually improved, meanwhile, the authenticity of the generated image is improved by adding an attention mechanism, and therefore the semantic consistency of the generated image and the text is improved.

Description

technical field [0001] The invention relates to the fields of natural language processing and computer vision, in particular to a method, device, computer equipment and storage medium for generating images from text. Background technique [0002] Both computer vision and natural language processing are used to process a single type of data, that is, images or text. Computer vision mainly focuses on the understanding of pictures, including sub-tasks such as image semantic segmentation, image classification, and target retrieval. Natural language processing mainly focuses on Modeling and processing of text information, including subtasks such as machine translation, named entity recognition, and word segmentation. In recent years, multimodal tasks that combine multiple data types such as images, texts, and videos have attracted more and more attention from researchers. It can link the relationship between various types of data, such as mapping and fusion. The two most common ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/583G06V10/74G06V10/82G06K9/62G06N3/04G06N3/08
CPCG06F16/5846G06N3/08G06N3/045G06F18/22
Inventor 陆璐叶锡洪冼允廷
Owner SOUTH CHINA UNIV OF TECH