A visual positioning method, device, equipment and medium

Abstract

The application discloses a visual positioning method, device, equipment and medium, and relates to the technical field of artificial intelligence. The method includes: performing feature splicing on image coding features and text coding features; and performing feature fusion on the spliced ​​coding features to obtain a first The post-fusion coding feature; based on the preset cross-attention mechanism, the noise correction is performed on the first post-fusion coding feature and the text coding feature respectively, and the post-fusion coding feature and post-modification text coding feature are obtained, and the post-splicing coding feature and the post-modification text coding feature are encoded The feature is fused to obtain the second fused coding feature; the preset frame feature is modified by using the target coding feature determined based on the revised fused feature and the second fused coding feature to predict the regional position coordinates of the target visual object, it can be seen that, The present application corrects image and text noise based on a preset cross-attention mechanism, reduces the influence of noise by reducing the attention to the noise part in the text, and realizes anti-noise visual positioning.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, and in particular, to a visual positioning method, device, equipment and medium. Background technique [0002] In recent years, Multi Modal (MM) has become a very important research direction in the field of artificial intelligence. Due to its emphasis on the fusion of vision, text, speech and other information, various algorithms related to multimodality are also emerging in an endless stream: various methods based on Convolutional Neural Networks (CNN) and attention mechanisms (attention) have their own A wide range of applications have become mainstream methods in fields such as Visual Commonsense Reasoning (VCR), Visual Question Answering (VQA), and Visual Grounding (VG). [0003] The visual localization task is one of the important research directions in the field of multimodal artificial intelligence. The task aims to locate the relevant object in the picture accord...

AI Technical Summary

Problems solved by technology

Under normal circumstances, human slips of the tongue, subjective deviation when describing objects, ambiguity of description sentences and other reasons will lead to errors in the text. These errors are very common in daily life, but they are very easy in the process of AI algorithm design. Ignored, this becomes an obstacle between existing methods and implementation

In short, when there are some errors in the input text, it is difficult for existing methods to find and locate the object that the sentence itself wants to describe.

Images