A method for generating images from text
By introducing the Transformer module and a multi-stage generation network, combining self-attention and global attention mechanisms, the problems of unreasonable images and unclear details in the existing text image generation method are solved, and the generated image details are clearer and the resolution is higher.
CN114022582BActive Publication Date: 2025-07-18ZHEJIANG UNIV OF TECH
2 Cites 2 Cited by
Patent Information
- Application Number
- CN202111109265.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-22
- Publication Date
- 2025-07-18
- Estimated Expiration
- 2041-09-22
AI Technical Summary
Technical Problem
The existing text image generation methods have problems such as unreasonable images and unclear details, especially the AttnGAN network-based methods still have shortcomings in the generated image quality and resolution.
Method used
A multi-stage generation method based on the Transformer module and AttnGAN network is adopted. By extracting the word features and sentence features of the text, combining random noise for conditional enhancement, and iterative training between multiple generators and discriminators, the self-attention mechanism and global attention module are used to improve image details and resolution.
Benefits of technology
The resulting image details are clearer, improving image quality and resolution.
✦ Generated by Eureka AI based on patent content.
Abstract
The present invention relates to a method for generating images from text. Based on the Transformer module and the AttnGAN network, after encoding the text through a text encoder, sentence features and word features are obtained. The sentence features are passed through a conditional enhancement module to obtain feature vectors, which are fused with random noise vectors and input into the Transformer module for learning. An improved feature vector is output and input into a generator to generate an initial image of 64*64 pixels roughly. The initial synthesized image and the improved feature vector are input into a discriminator for discrimination, and the generator is trained according to a loss function. Sequentially, the improved feature vector from the previous step and the word features are input into a neural network for upsampling to obtain a fused vector, which is then input into the generator to obtain images of 128*128 pixels and 256*256 pixels. The images generated by the present invention have clearer detail contours compared to the images generated by the previous traditional AttnGAN method.
Need to check novelty before this filing date? Find Prior Art