Real-time visual target tracking method based on twin convolutional network and long short-term memory network

Abstract

The invention relates to a real-time visual target tracking method based on a twin convolutional network and a long short-term memory network, which comprises the following steps of: firstly, for a video sequence to be tracked, taking two continuous frames of images as inputs acquired by the network each time; carrying out feature extraction on two continuous frames of input images through a twinconvolutional network, obtaining appearance and semantic features of different levels after convolution operation, and combining depth features of high and low levels through full-connection cascading; transmitting the depth features to a long-term and short-term memory network containing two LSTM units for sequence modeling, performing activation screening on target features at different positions in the sequence by an LSTM forgetting gate, and outputting state information of a current target through an output gate; and finally, receiving a full connection layer output by the LSTM to output the predicted position coordinates of the target in the current frame, and updating the search area of the target in the next frame. The tracking speed is greatly improved while certain tracking stability and accuracy are guaranteed, and the tracking real-time performance is greatly improved.

Description

technical field

[0001] The invention belongs to the technical field of computer vision and visual target tracking, and in particular relates to a real-time visual target tracking method based on twin convolutional networks and long-short-term memory networks. Background technique

[0002] Visual object tracking technology is one of the most important problems in the field of computer vision, and has a wide range of application scenarios, such as security monitoring, smart home, smart city, etc. Its main task is to find the position of the same target in the current frame from the second frame onwards given the position and state of the target to be tracked in the first frame in a set of video sequences. Although target tracking technology has been extensively studied, the stability, accuracy and speed of visual target tracking are still difficult due to interference factors such as target deformation, background occlusion, motion blur, and illumination changes that may occur...

Images