A task-oriented text generation image network model

Abstract

The invention discloses a task-oriented text generation image network model, and aims to provide a text generation image network model for generating high-quality images consistent with text semanticsand rich in content. The invention is realized through the following technical scheme: a common sense reasoning module enriches and expands entities and entity attributes in natural language text description, and then respectively constructs an entity relationship scene graph and entity attribute semantic vectors for the natural language text description; the global generation model inputs the entity relationship scene graph into a GCN, respectively inputs the obtained embedded vectors into a mask regression network and a bounding box regression network, estimates a segmentation mask and a bounding box of each entity, fuses all entity layouts to form a scene layout, and generates an initial image in combination with a convolutional neural network; and the local refinement model takes theentity attribute semantic vector and the initial image feature mapping as input, and generates a high-quality image with rich and harmonious content in combination with the RRRN and the convolutionalneural network.

Description

technical field [0001] The present invention relates to the field of computer science, especially the Generative Adversarial Networks (GAN) technology in the field of deep learning, in particular to a task-oriented text generation image network model. Background technique [0002] With the widespread use of camera-capable mobile smart terminals and the rapid development of the Internet, multi-modal data that integrates visual and textual information is increasing dramatically, for example, photos with text annotations, graphic-text content in newspaper articles, captioned Video and multimodal interaction data appearing in social media. Image text description method can effectively organize image data, and combine text information retrieval technology to search massive image data conveniently. In addition, using the image-to-text description method can not only understand the content of the speaker from the images in the slides, but also help the visually impaired to underst...

AI Technical Summary

Problems solved by technology

[0006] The object of the present invention is to provide a method for generating high-quality images that are consistent with text semantics, rich in content, and harmonious in order to overcome the above-mentioned deficiencies in the prior art The task-oriented text generation image network model solves the problems of poor ability to deal with multiple entities and complex relationships between multiple entities, low resolution of generated images, and poor quality of details such as textures and edges in the prior art

Images