RGBT visual tracking method and system based on two-stage fusion structure search

Abstract

The invention discloses an RGBT visual tracking method and system based on two-stage fusion structure search, belongs to the technical field of computer vision, and solves the problem of how to find an optimal fusion network structure for RGBT tracking based on robust instance representation so as to further improve the tracking performance. In order to extract shared feature representation having robustness to various challenges such as illumination variation, movement blur and scale variation, a multi-domain learning framework is introduced to search for a fusion network structure in a general fusion space offline. In the online tracking stage, the fusion structure of each video sequence is searched online in the fusion space perceived by the instance so as to cope with challenges specific to the instance; the two-stage search algorithm can dynamically update a video fusion strategy, so that an appropriate fusion network structure is found for RGBT tracking based on robust instance representation, and the tracking performance is further improved.

Description

technical field

[0001] The invention belongs to the technical field of computer vision, and relates to a RGBT visual tracking method and system based on two-stage fusion structure search. Background technique

[0002] Object tracking is a hot issue in the field of computer vision. Object tracking is also one of the key technologies for unmanned driving, intelligent transportation and intelligent surveillance. Object tracking is to estimate the position of the object in subsequent frames given the bounding box of the initial frame. Most of the current tracking algorithms are based on the single-mode condition of visible light, which will be greatly affected under some extreme conditions, such as severe weather and strong changes in illumination, etc. The performance of single-mode tracking algorithms is often unsatisfactory. The modal fusion tracking of visible light and thermal infrared is called RGBT (Red Green Blue Thermal) tracking. Since visible light information and t...

Examples

Embodiment 1

[0037] Such as figure 1 , 2As shown, a RGBT visual tracking method based on two-stage fusion structure search, which specifically includes two stages of offline search and online tracking:

[0038] Such as figure 1 As shown, a general search space is designed in the offline search stage, including different fusion methods of the VGG-M convolutional layer, and five activation functions Tanh, ReLU, PReLU, LReLU, and ReLU6. The size of the search space is exponentially related to the number of possible fusion layers, so the search space is gradually explored according to the number of fusion layers, which is consistent with the idea of ​​​​progressive neural structure search, starting from a simple fusion layer of 1, and sequentially expanding the fusion number of layers. Train a proxy function to further guide the exploration of the search space. In order to learn the commonality of targets in different videos, a multi-domain learning training method is used. Assume that K vi...

technical field

[0001] The invention belongs to the technical field of computer vision, and relates to a RGBT visual tracking method and system based on two-stage fusion structure search. Background technique

[0002] Object tracking is a hot issue in the field of computer vision. Object tracking is also one of the key technologies for unmanned driving, intelligent transportation and intelligent surveillance. Object tracking is to estimate the position of the object in subsequent frames given the bounding box of the initial frame. Most of the current tracking algorithms are based on the single-mode condition of visible light, which will be greatly affected under some extreme conditions, such as severe weather and strong changes in illumination, etc. The performance of single-mode tracking algorithms is often unsatisfactory. The modal fusion tracking of visible light and thermal infrared is called RGBT (Red Green Blue Thermal) tracking. Since visible light information and t...

Examples

Embodiment 1

[0037] Such as figure 1 , 2As shown, a RGBT visual tracking method based on two-stage fusion structure search, which specifically includes two stages of offline search and online tracking:

[0038] Such as figure 1 As shown, a general search space is designed in the offline search stage, including different fusion methods of the VGG-M convolutional layer, and five activation functions Tanh, ReLU, PReLU, LReLU, and ReLU6. The size of the search space is exponentially related to the number of possible fusion layers, so the search space is gradually explored according to the number of fusion layers, which is consistent with the idea of ​​​​progressive neural structure search, starting from a simple fusion layer of 1, and sequentially expanding the fusion number of layers. Train a proxy function to further guide the exploration of the search space. In order to learn the commonality of targets in different videos, a multi-domain learning training method is used. Assume that K vi...

Images