Picture description generation method and system based on Actor-Critic generative adversarial network

Abstract

The invention discloses a picture description generation method and system based on an Actor-Critic generative adversarial network, and the method comprises the following steps: (1) obtaining a picture described by a known text, carrying out the preprocessing, and constructing a training set, (2) establishing a target network based on a generative adversarial network and an Actor-Critic algorithm,wherein the target network comprises a generator network, a discriminator network and a Critic network, (3) inputting the pictures in the training set and the text description of the pictures into the target network, performing pre-training and adversarial training on the generator and the discriminator, and performing single-step updating on the parameters of the generator by adopting an Actor-Critic algorithm, and (4) inputting the target picture needing to generate the text description into the trained generator to obtain the text description of the target picture. According to the method,based on the Actor-Critic algorithm, an adversarial network technology is adopted, and diversified text description can be generated on a given image.

Description

technical field [0001] The present invention relates to the fields of reinforcement learning, computer vision and natural language processing, in particular to a method and system for generating picture descriptions based on Actor-Critic generative confrontation networks. Background technique [0002] Image summarization (Image Caption) is an important problem in computer vision. [0003] In traditional image summarization tasks, the commonly used model is the Encoder-Decoder (encoding-decoding) model. In general, Encoder uses Convolutional Neural Networks (CNN) to extract image features, while Decoder uses Recursive Neural Network (RNN) to generate summaries from image features. This is because CNN performs well in extracting image features, while RNN has a memory ability for input and is outstanding in text generation. [0004] Proposed in a paper published by Qi Wu (Q.Wu, C.Shen, A.van den Hengel, L.Liu, and A.Dick.What Value DoExplicit High-Level Concepts Have in Visio...

AI Technical Summary

Problems solved by technology

[0007] SeqGAN solves the training problem of discrete data through reinforcement learning, and provides a set of methods for GAN in the field of text generation, but many experimental results show that the quality of the text generated by the SeqGAN model is rough, far below expectations

Images