Differential description statement generation method and device, equipment and medium

Abstract

The invention discloses a difference description statement generation method and device, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the feature splicing of image coding features and text coding features, inputting the spliced coding features into a preset image-text alignment unit constructed based on a preset self-attention mechanism to obtain spliced alignment features, using a preset noise monitoring unit constructed based on a preset self-attention mechanism and a preset cross-attention mechanism to process image alignment features and text alignment features obtained after splitting the text coding features and the spliced alignment features so as to extract difference signals; the difference description statement is generated by utilizing the preset difference description generation algorithm and based on the difference signal, and therefore, the part, which cannot be aligned with the image, in the human language text is positioned based on the preset cross-attention mechanism, and the corresponding interpretation description is given, so that the problem that a computer cannot perform normal reasoning due to human language errors is solved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a method, device, equipment and medium for generating difference description sentences. Background technique [0002] In recent years, multimodality has become an emerging research direction in the field of artificial intelligence, such as visual commonsense reasoning (Visual Commonsense Reasoning, VCR), visual question answering (Visual Question Answering, VQA) and other fields have become key research directions in the industry. In the field of multimodality, existing topics are based on the assumption that human language is infallible in multimodal processes, i.e., human language will necessarily match images. However, for humans, slips of the tongue are unavoidable, although usually, human language errors are not very outrageous, that is to say, the text and the image itself are relatively close, but the wrong use of a certain subject or attributive wi...

AI Technical Summary

Problems solved by technology

[0004] In view of this, the purpose of the present invention is to provide a method, device, device and medium for generating difference description sentences, which can solve the problem that the computer cannot reason normally due to human language errors, and further enhance the human-computer interaction experience. The specific solution is as follows :

Images