Image natural language description generation method and device with cross-linguistic learning ability

Abstract

The invention provides an image natural language description generation method and device with cross-linguistic learning ability. The method comprises the steps that English description sentences are translated into target language description sentences through a machine; part of the target language description sentences are selected randomly to form a training sample set; a smooth sample set and a non-smooth sample set are used for training a sentence smooth degree model; the target language description sentences in a candidate data set are subjected to smooth degree evaluation through the sentence smooth degree model, and according to the smooth degree probability of each target language description sentence, a strategy for training an image description sentence generation model is set; according to the strategy, the image description sentence generation model is trained, and the trained image description sentence generation model is obtained. According to the image natural language description generation method and device with the cross-linguistic learning ability, the aim is achieved that the image description sentence generation model of a target language is guided, trained and generated dependent on smooth degree evaluation results, the influence of non-smooth target language description sentences on the training process is lowered, and the accuracy of the image description sentence generation model of the target language is improved.

Description

technical field [0001] The invention relates to the technical field of image recognition, in particular to a method and device for generating natural language descriptions of images with cross-language learning capabilities. Background technique [0002] Image captioning refers to that given a picture, the computer automatically generates a natural sentence that can describe the main visual content of the corresponding picture. The natural sentence is used to describe the specific behavior of the main object in the corresponding picture in a specific scene. Image description sentences can contain more semantic information than traditional image description tags, which is helpful for better management of image data. [0003] However, at present, the description sentences of large image sets are generally described in English. When it comes to cross-language image description, it is necessary to use manual annotation to collect training data in the corresponding language for ...

AI Technical Summary

Problems solved by technology

[0003] However, at present, the description sentences of large image sets are generally described in English. When it comes to cross-language image description, it is necessary to use manual annotation to collect training data in the corresponding language for the target language, or translate the English description sentences through a translation machine. Describing sentences for the target language, these methods have high operating costs, low efficiency of image language description, and poor readability

Images