Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Target tracking method, system, device and medium based on two-stream convolution neural network

A convolutional neural network and neural network technology, applied in image data processing, instruments, computing, etc., can solve problems such as lack of grasp of motion information, insufficient video timing, lack of large amounts of data, etc., to achieve good network generalization ability, Universal and universal, accurate tracking effect

Active Publication Date: 2019-03-01
SOUTH CHINA UNIV OF TECH
View PDF4 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The disadvantage of the method based on deep learning is that due to the particularity of the target tracking task, only the label data of the first frame of the picture is provided, and there is a lack of large amounts of data to train the neural network. The general practice is to migrate the model trained on the large-scale classification image data set to target tracking. Come, that is, a region-based target detection method, there is no sufficient timing of the video, and it is not sure to track the motion information between the target frames

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Target tracking method, system, device and medium based on two-stream convolution neural network
  • Target tracking method, system, device and medium based on two-stream convolution neural network
  • Target tracking method, system, device and medium based on two-stream convolution neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0075] The three-dimensional convolutional neural network is a kind of convolutional neural network, which originated in the fields of motion, body, and gesture detection. It is different from the two-dimensional convolutional neural network commonly used in the field of image classification and detection. It adds a time dimension, so It has excellent time-series feature expression ability, and was later introduced into the field of video classification and retrieval.

[0076] Different from tasks such as image classification, visual target tracking tasks not only need to extract the features of the target itself, but also need to extract the motion change information of the target between video frames, that is, sequence features. The present invention provides a target tracking method based on a two-stream convolutional neural network. The method firstly applies a three-dimensional convolutional neural network to the field of visual target tracking, and combined with a two-dim...

Embodiment 2

[0122] Such as Figure 7As shown, this embodiment provides a target tracking system based on a two-stream convolutional neural network, which includes a first building block 701, a second building block 702, an additive fusion module 703, a third building block 704, and a bounding box The regression module 705, the offline training module 706 and the online fine-tuning module 707, the specific functions of each module are as follows:

[0123] The first building block 701 is used to build a spatial flow two-dimensional convolutional neural network to extract feature information of image blocks in the current frame. The first building block 701 is as follows: Figure 8 shown, including:

[0124] The first input unit 7011 is configured to perform Gaussian sampling of S image blocks in the current frame based on the target neighborhood in the previous frame of the current frame, as the input of the spatial flow two-dimensional convolutional neural network; wherein, the spatial fl...

Embodiment 3

[0149] This embodiment provides a computer device, which may be a desktop computer, which includes a processor, a memory, a display, and a network interface connected through a system bus, the processor of the computer device is used to provide computing and control capabilities, the computer The memory of the device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs and databases. The internal memory is the operating system and computer programs in the non-volatile storage medium. An environment is provided, and when the processor executes the computer program stored in the memory, the target tracking method of the above-mentioned embodiment 1 is realized, as follows:

[0150] Construct a spatial flow two-dimensional convolutional neural network to extract the feature information of the image block in the current frame;

[0151] Construct a time series flow three-dimensional convolutiona...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a target tracking method, system, device and medium based on a dual-stream convolution neural network. The method comprises the following steps: a spatial stream two-dimensional convolution neural network is constructed to extract the characteristic information of an image block in a current frame; a three-dimensional convolutional neural network is constructed to extract the motion information of the objects between frames in a video sequence within a certain time range; additive fusion of feature information of spatial flow two-dimensional convolution neural network and timing sequence flow three-dimensional convolution neural network; according to the fused feature information, a fully connected sub-network is constructed to extract the image blocks that meet therequirements. Boundary box regression is used to get the predicted position and size of the object in the current frame. Two-dimensional convolution neural network of spatial flow and three-dimensional convolution neural network of temporal flow are trained off-line before target tracking. In the process of target tracking, the on-line fine tuning of all-connected subnetworks is carried out. Theinvention achieves good tracking effect.

Description

technical field [0001] The invention relates to a target tracking method, in particular to a target tracking method, system, computer equipment and storage medium based on a dual-stream convolutional neural network, belonging to the field of target tracking of computer vision. Background technique [0002] Visual target (single target) tracking task has been a research hotspot in the field of computer vision and has been widely used, especially in recent years with the rapid development of scientific and technological productivity, video surveillance, drone flight, automatic driving and other fields are in urgent need of excellent target tracking algorithm. [0003] The visual target tracking task describes that in a given video sequence scene, only the position of the target in the first frame is provided, and then the next position and size of the target is predicted by the algorithm. Although a large number of algorithms have emerged in recent years, there is still no go...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T7/20
CPCG06T7/207G06T7/246G06T2207/10016G06T2207/20081G06T2207/20084
Inventor 黄双萍伍思航李豪杰
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com