RGBT visual tracking method and system based on two-stage fusion structure search

Abstract

The invention discloses an RGBT visual tracking method and system based on two-stage fusion structure search, belongs to the technical field of computer vision, and solves the problem of how to find an optimal fusion network structure for RGBT tracking based on robust instance representation so as to further improve the tracking performance. In order to extract shared feature representation having robustness to various challenges such as illumination variation, movement blur and scale variation, a multi-domain learning framework is introduced to search for a fusion network structure in a general fusion space offline. In the online tracking stage, the fusion structure of each video sequence is searched online in the fusion space perceived by the instance so as to cope with challenges specific to the instance; the two-stage search algorithm can dynamically update a video fusion strategy, so that an appropriate fusion network structure is found for RGBT tracking based on robust instance representation, and the tracking performance is further improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and relates to a RGBT visual tracking method and system based on two-stage fusion structure search. Background technique [0002] Object tracking is a hot issue in the field of computer vision. Object tracking is also one of the key technologies for unmanned driving, intelligent transportation and intelligent surveillance. Object tracking is to estimate the position of the object in subsequent frames given the bounding box of the initial frame. Most of the current tracking algorithms are based on the single-mode condition of visible light, which will be greatly affected under some extreme conditions, such as severe weather and strong changes in illumination, etc. The performance of single-mode tracking algorithms is often unsatisfactory. The modal fusion tracking of visible light and thermal infrared is called RGBT (Red Green Blue Thermal) tracking. Since visible light information and t...

AI Technical Summary

Problems solved by technology

First, these manually designed RGBT trackers require extensive repeated experiments, expert experience, and scientific intuition

Second, these fusion strategies ignore the potential benefits of cross-layer fusion

Finally, due to the fixed structure, these trackers are usually difficult to deal with various challenges in the tracking process

[0004] The existing methods have the following disadvantages: 1) The manually designed fusion network requires a lot of repeated experiments, expert experience and scientific intuition, the fusion of different convolutional layers has different effects, and finding an optimal fusion structure requires a lot of manpower 2) Since the structure of the hand-designed model is fixed, it is difficult to deal with various challenges in tracking

Images