Target tracking method based on capsule network and natural language query

Abstract

The invention discloses a target tracking method based on a capsule network and natural language query, and the method comprises the following steps: 1, giving a search region of a current frame and a corresponding text query, sending the search region into a visual encoder to extract the feature representation of vision, and sending the text query into a text encoder to extract the feature representation of a text; 2, constructing a visual capsule by using the visual feature representation extracted by the visual encoder, constructing a text capsule by using the text feature representation extracted by the text encoder, and designing a visual-text routing module and a text-visual routing module on the basis of the visual capsule and the text capsule; and 3, serially connecting the output of the vision-text routing module and the output of the text-vision routing module, and generating a response diagram of the target through a decoder. The tracker is initialized only by using the natural language, the method can be close to other methods, and meanwhile, the result obtained by performing initialization by using the natural language query and the initial bounding box has good robustness.

Description

technical field [0001] The invention relates to a target tracking method, in particular to a target tracking method based on capsule network and natural language query. Background technique [0002] Recently, Tracking by Natural Language Query (TNL) has gained increasing attention because it does not require a hand-labeled rectangular box to initialize a tracker to locate an object of interest. The combination of natural language understanding and visual target tracking has the following advantages: first, this combination breaks through the limitation of manually marking the initial frame; second, by jointly optimizing the two heterogeneous features of vision and language, the tracking method can be significantly improved in the long-term. The ability to accurately locate targets during tracking. However, previous methods have not achieved competitive results on the dataset, and there are two problems: (1) the alignment strategy between text query and video frame is too si...

AI Technical Summary

Problems solved by technology

However, previous methods have not achieved competitive results on the dataset, and there are two problems: (1) the alignment strategy between text query and video frame is too simple; (2) 2D convolution-based model decoding The latter representations often fail to capture the relationships between different entities and part-entities

Images