Image description generation method and system based on unsupervised uniqueness optimization

Abstract

The invention discloses an image description generation method and system based on unsupervised uniqueness optimization, and relates to the technical field of image description, and the method comprises the steps: S1, obtaining paired images and real sentence description generated by human beings, and storing the paired images and the real sentence description in a training set; s2, training an image description retrieval model by using paired data contained in the training set; s3, constructing an image description generation model; s4, acquiring images of the training set and inputting the images into an image description generation model, enabling the image description generation model to generate sentence description corresponding to the images, and taking the images and the generatedsentence description as paired data to be stored in a comparison set; s5, estimating the similarity of paired data in the comparison set by utilizing an image description retrieval model to obtain loss and gradient; and S6, adjusting the image description generation model according to the loss and the gradient, and returning to execute the step S4 or outputting a final image description generationmodel. According to the method, the high-quality sentence description of the image can be generated in an unsupervised manner, and the description has diversity and uniqueness.

Description

technical field [0001] The invention relates to the technical field of image description, in particular to an image description generation method and system based on unsupervised uniqueness optimization. Background technique [0002] Image description is a task that requires models to acquire a multimodal understanding of the world and express this understanding in natural language text, making it relevant to various fields from human-computer interaction to data management. The practical goal is to automatically generate description images. Related natural language descriptions. Most of the latest neural network models are built on an encoder-decoder architecture, where a convolutional neural network (CNN) is used as an encoder for image features, which are fed to a recurrent neural network (RNN), which The network generates sentence descriptions by acting as a decoder. Decoders typically also include one or more attention layers to focus sentence descriptions on the most...

AI Technical Summary

Problems solved by technology

Methods based on generative adversarial networks can generate diverse descriptions, but due to the complexity of generative adversarial networks, these methods often have the problem of unstable training

[0004] To sum up, the existing image description methods based on maximum likelihood estimation tend to generate sentences similar to the training set while ignoring specific image details, resulting in the lack of diversity and vividness of the generated sentence descriptions, which are different from human-generated descriptions. larger

Images