A cross-modal fusion target tracking method

Abstract

The invention relates to the field of computer information technology and provides a cross-modal fusion target tracking method. The purpose is to solve the problem that the difference between cross-modal targets is too large, and it is difficult to obtain better results simply by using feature-based matching. The main scheme includes building and generating an adversarial neural network composed of a pixel alignment module, a feature alignment module, and a joint discriminant module, training and generating an adversarial network on a data set, extracting targets to be recognized from videos collected by different cameras, and inputting the trained joint The discriminant module obtains the feature similarity between the target and all targets to be identified; uses the transfer time data set of the marked target between the cameras to train a logistic regression model that predicts the temporal similarity between targets according to the transfer time, The model is used to calculate the time similarity between two targets; the total similarity is obtained by adding the feature similarity and time similarity, and the target pair with the highest total similarity is the same target.

Description

technical field [0001] The invention relates to the field of computer information technology, and provides a cross-modal fusion target tracking method. Background technique [0002] An RGB image has three channels containing color information for visible light, while an IR image has one channel containing information for invisible light. Therefore, even for humans, it is difficult to recognize people well by using color information. To address this problem, existing cross-modal re-ID methods mainly focus on bridging the gap between RGB and IR images through feature alignment, such as figure 2 shown. The basic idea is to match real RGB and IR images through feature representation learning. Due to the large cross-modal difference between the two modalities, it is difficult to directly match RGB and IR images in a shared feature space. [0003] Different from the existing methods by directly matching RGB and IR images, the heuristic method is to generate a pseudo-IR image b...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the problem that the difference between cross-modal targets is too large, and it is difficult to obtain better results by simply using feature-based matching