Cross-modal fusion target tracking method

Abstract

The invention relates to the technical field of computer information, and provides a cross-modal fusion target tracking method. The objective of the invention is to solve the problem that the difference between cross-modal targets is too large and a good effect is difficult to obtain by purely using feature-based matching. According to the main scheme, the method comprises the steps of constructing and generating an adversarial neural network composed of a pixel alignment module, a feature alignment module and a joint discrimination module, training on a data set to generate the adversarial network, extracting to-be-recognized targets from videos collected by different cameras, inputting the to-be-recognized targets into the trained joint discrimination module, and obtaining the feature similarity between the target and all the to-be-recognized targets; training a logistic regression model for predicting the time similarity between the targets according to the transfer time by using the transfer time data set of the marked targets between the cameras, and calculating the time similarity between the two targets by using the model; and adding the feature similarity and the time similarity to obtain total similarity, and taking the target pair with the highest total similarity as the same target.

Description

technical field [0001] The invention relates to the field of computer information technology, and provides a cross-modal fusion target tracking method. Background technique [0002] An RGB image has three channels containing color information for visible light, while an IR image has one channel containing information for invisible light. Therefore, even for humans, it is difficult to recognize people well by using color information. To address this problem, existing cross-modal re-ID methods mainly focus on bridging the gap between RGB and IR images through feature alignment, such as figure 2 shown. The basic idea is to match real RGB and IR images through feature representation learning. Due to the large cross-modal difference between the two modalities, it is difficult to directly match RGB and IR images in a shared feature space. [0003] Different from the existing methods by directly matching RGB and IR images, the heuristic method is to generate a pseudo-IR image b...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the problem that the difference between cross-modal targets is too large, and it is difficult to obtain better results by simply using feature-based matching

Images