Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A text generation image method and system based on transition space mapping

A technology for image generation and spatial mapping, applied in 2D image generation, image data processing, texture/color, etc., can solve the problems of limiting the effect of generated images, ignoring text and image related information, etc., to improve quality and enhance semantics consistent effect

Active Publication Date: 2021-11-16
PEKING UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Although the above methods based on generative adversarial networks can achieve certain results, because the generative network has many network layers, the training only relies on a single adversarial loss function to constrain the consistency between text and images, ignoring the differences between text and images. A large amount of related information between them limits the effect of generating images

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A text generation image method and system based on transition space mapping
  • A text generation image method and system based on transition space mapping
  • A text generation image method and system based on transition space mapping

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0021] The text generation image method based on the transition space mapping of the present invention, its flow process is as follows figure 1 shown, including the following steps:

[0022] (1) Using images and texts from the database, train a text-to-image model consisting of a cascade of a transition space mapping network and a generative adversarial network.

[0023] The procedure of this step is as figure 2 shown. The present invention designs a transition space mapping network M, which is composed of multiple layers of fully connected layers, and can combine random noise z to input text Mapping to transition spaces yields interpretable feature representations Then, the present invention expresses the interpretable features Input to the generator G in the generative confrontation network, after the multi-layer convolutio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a text generation image method and system based on transition space mapping. The method consists of the following steps: 1. Using images and text from a database, train a text-to-image model consisting of a cascade of a transition space mapping network and a generative adversarial network. 2. For the text input by the user, use the trained text to generate an image model to generate an image that matches the content of the input text. Compared with the existing methods, the invention can significantly improve the quality of the generated image and enhance the semantic consistency between the generated image and the input text.

Description

technical field [0001] The invention relates to the field of image generation, in particular to a text generation image method and system based on transition space mapping. Background technique [0002] In recent years, with the rapid development of the Internet and multimedia technology, the total amount of multimedia data has continued to grow and has become the main content of big data. People usually use some traditional computer vision methods to process large amounts of information, such as common image classification. However, these methods can only provide users with limited information. For example, image classification can only provide category labels with little information. Therefore, methods with data generation capabilities emerge as the times require. They can not only provide more samples to better meet user needs, but also allow for flexible creation and are easier for users to use. Image generation from text means that the user provides a text description...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06T11/00
CPCG06T11/001
Inventor 彭宇新袁明宽
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com