Unlock instant, AI-driven research and patent intelligence for your innovation.

A detail-preserving image generation method and system based on text semantics

An image generation, text image technology, applied in semantic analysis, neural learning methods, biological neural network models, etc., can solve problems such as inaccuracy, rough detail correction, model inability to generate visual attributes, etc. Granularity and matching degree, the effect of ensuring accuracy

Active Publication Date: 2022-07-05
SHANDONG NORMAL UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, to achieve effective image processing guided by text descriptions by fusing text and image information, it is necessary to utilize both text and image cross-modal information to generate new attributes that match the given text. Existing methods usually choose to directly Combining image and global sentence features, it is impossible to accurately associate fine-grained words with corresponding visual attributes that require detail correction, resulting in inaccurate and coarse detail correction
For example, the model cannot generate detailed visual attributes such as eye circle color, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A detail-preserving image generation method and system based on text semantics
  • A detail-preserving image generation method and system based on text semantics
  • A detail-preserving image generation method and system based on text semantics

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0037] In order to accurately convert text into images, in this embodiment, a method for generating a detail-preserving image based on text semantics is disclosed, including:

[0038] get text information;

[0039] Extract text features, sentence features and word features of text information;

[0040] Input text features, sentence features and word features into the trained image generation adversarial network, and output text images;

[0041] Among them, the generation network in the image generation adversarial network includes a multi-stage image feature conversion network, and a detail optimization module is added to each stage of the network. The detail optimization module optimizes the hidden features of the network at each stage, and outputs hidden visual features. In the visual feature input generator, the synthetic image is output, and the hidden visual features output by the detail optimization module of the network in the remaining stages except the last stage are...

Embodiment 2

[0090] In this embodiment, a text semantics-based detail-preserving image generation system is disclosed, including:

[0091] Text information acquisition module, used to acquire text information;

[0092] Feature extraction module, used to extract text features, sentence features and word features of text information;

[0093] The text image acquisition module is used to input text features, sentence features and word features into the trained image generation adversarial network, and output text images;

Embodiment 3

[0095] In this embodiment, an electronic device is disclosed, which includes a memory, a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, one of the methods disclosed in Embodiment 1 is completed. A text semantics-based detail-preserving image generation method described the steps.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A method and system for generating a detail-preserving image based on text semantics disclosed in the present disclosure include: acquiring text information; extracting text features, sentence features and word features of the text information; inputting the text features, sentence features and word features into trained In the image generative adversarial network, text images are output; among them, the image generative adversarial network includes a generative network based on a hybrid attention module and a detail optimization module and a discriminative network based on a deep attention polymorphic similarity model. The accuracy of the generated text images is guaranteed.

Description

technical field [0001] The invention relates to the technical field of cross-modal text generation image, in particular to a method and system for generating a detail-preserving image based on text semantics. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] Generating highly fine-grained images that semantically match a given textual description is a challenging problem and has huge potential applications, including photo editing, image inpainting, image colorization, style transfer, computer-aided design, and more. Recently, text-to-image generation has made significant progress thanks to the proposal of Generative Adversarial Networks (GANs). [0004] Among the methods of generating images from text, the Generative Adversarial Network (GAN) proposed by Reed et al. greatly improved the effect of generating images from text, and became th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/30G06F40/216G06K9/62G06N3/04G06N3/08G06V10/774G06V10/82
CPCG06F40/30G06F40/216G06N3/08G06N3/045G06F18/214
Inventor 刘丽马跃崔怀磊王泽康张化祥冯珊珊
Owner SHANDONG NORMAL UNIV