Text image generation method based on target detection

A technology for image generation and target detection, which is applied in 2D image generation, image data processing, neural learning methods, etc., can solve the problems of lack of attention mechanism and image quality reduction, so as to improve semantic consistency, improve training efficiency, The effect of improving quality

Pending Publication Date: 2021-09-07
UNIV OF SCI & TECH OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But when generating complex images, the lack of attention mechanism will reduce the quality of the image

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text image generation method based on target detection
  • Text image generation method based on target detection
  • Text image generation method based on target detection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Aiming at the problems existing in the prior art, the present invention proposes a method for generating an image based on a single-stream text of target detection. The text encoder encodes the text to obtain the feature vector of the entire sentence and the feature vector of each word, and constructs a method with a generated The generative model of the generator and the discriminator, introduces the attention mechanism in the generator and makes full use of sentence features and word features, introduces the target detection model in the discriminator to extract the visual features of each object in the image, and then combines word features and real images The visual features of each object in the text are matched, so as to achieve the goal of improving image quality and semantic consistency of text images.

[0037] The present invention only relies on text to generate visually real high-resolution images, ensures the semantic consistency of text and images, and is de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a text image generation method based on target detection. The method comprises the following steps: inputting a descriptive text; encoding by using a text encoder to obtain a feature vector of each word and a feature vector of the whole sentence; and outputting a vivid image consistent with text semantics through a trained generation model, wherein the generation model is a generative adversarial network model and comprises a generator and a discriminator, an attention mechanism is added in the generator, and the discriminator realizes refined discrimination of each target object in the image based on target detection. The generation model of the technology only comprises one generator and one discriminator, and the training efficiency of the model is improved while the quality of the generated image is ensured.

Description

technical field [0001] The invention relates to the fields of computer vision, natural language processing and generating confrontation networks, in particular to a method for generating images based on target detection text. Background technique [0002] Text-to-image generation is a hot issue in the field of computer vision, which aims to generate semantically relevant realistic images based on a descriptive text sentence, and has great application potential in image editing, video games, and computer-aided design. At present, the most classic and cutting-edge text generation image technology uses generative confrontation network (GAN) as a generative model. They first encode natural language text into text feature vectors. The generator of the GAN network generates images based on this, and the discriminator passes through Image features are extracted to distinguish generated images from real images, and then the loss function backpropagation alternately trains the genera...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T11/00G06K9/62G06F40/30G06N3/04G06N3/08
CPCG06T11/001G06F40/30G06N3/08G06N3/044G06N3/045G06F18/22
Inventor 杨雨嫣谢海永吴曼青
Owner UNIV OF SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap