Visual positioning method, device, equipment and medium

Abstract

The invention discloses a visual positioning method and device, equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the feature splicing of image coding features and text coding features; performing feature fusion on the spliced coding features to obtain first fused coding features; performing noise correction on the first fused coding feature and the text coding feature based on a preset cross-attention mechanism to obtain a corrected fused feature and a corrected text coding feature, and performing feature fusion on the spliced coding feature and the corrected text coding feature to obtain a second fused coding feature; and correcting the preset frame feature by using a target coding feature determined on the basis of the corrected fusion feature and the second fused coding feature to predict the regional position coordinate of the target visual object, so that the image-text noise is corrected on the basis of the preset cross-attention mechanism, and the image-text noise prediction accuracy is improved. The influence of noise is weakened by reducing the attention on the noise part in the text, and anti-noise visual positioning is realized.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a visual positioning method, device, equipment and medium. Background technique [0002] In recent years, Multi Modal (MM) has become a very important research direction in the field of artificial intelligence. Due to its emphasis on the fusion of vision, text, voice and other information, various algorithms related to multimodality emerge in endlessly: various methods based on convolutional neural networks (CNN) and attention mechanisms (attention) have their own advantages. It has a wide range of applications and has become a mainstream method in fields such as Visual Commonsense Reasoning (VCR), Visual Question Answering (VQA), and Visual Grounding (VG). [0003] The visual localization task is one of the important research directions in the field of multimodal artificial intelligence. This task aims to locate the relevant object in the picture accordi...

AI Technical Summary

Problems solved by technology

Under normal circumstances, human slips of the tongue, subjective deviation when describing objects, ambiguity of description sentences and other reasons will lead to errors in the text. These errors are very common in daily life, but they are very easy in the process of AI algorithm design. Ignored, this becomes an obstacle between existing methods and implementation

In short, when there are some errors in the input text, it is difficult for existing methods to find and locate the object that the sentence itself wants to describe.

Images