The invention discloses a video target tracking method based on multi-source heterogeneous
data collaboration, and belongs to the technical field of
computer vision. The method aims at solving the problems that existing single-mode tracking robustness is insufficient and multi-mode fusion efficiency is low. The method comprises the following core steps: firstly, carrying out adaptive
cutting on a red-green-blue (RGB) image and a
depth map which are synchronously acquired, and constructing six-channel input based on pseudo-
color mapping and channel splicing of a JET
color table; secondly, respectively extracting independent feature tokens of RGB and depth
modes by adopting a double-
branch embedded structure; furthermore, a visual prompt module is embedded in each layer of the visual Transform
backbone network, so that hierarchical progressive fusion of two
modal features is realized; meanwhile, deploying candidate
elimination modules in the third layer, the sixth layer and the ninth layer, and dynamically screening high-confidence-coefficient feature tokens based on the attention weight of a template-search area so as to reduce the calculation overhead; then, a center
heat map, a size map and an offset map of the target are predicted in parallel through a center positioning head structure, and an accurate bounding box is generated; and finally, performing end-to-end training in combination with a multi-task
loss function, executing a complete process including initialization and tracking in a reasoning stage, and assisting with a post-
processing strategy to improve stability. According to the method, the geometric advantages of depth information and the
global modeling capability of Transform are fully utilized, and the tracking precision, robustness and reasoning efficiency in a complex scene are remarkably improved.