Target tracking method, system, device and medium based on two-stream convolution neural network

A convolutional neural network and neural network technology, applied in image data processing, instruments, computing, etc., can solve problems such as lack of grasp of motion information, insufficient video timing, lack of large amounts of data, etc., to achieve good network generalization ability, Universal and universal, accurate tracking effect

Active Publication Date: 2019-03-01
SOUTH CHINA UNIV OF TECH
View PDF4 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The disadvantage of the method based on deep learning is that due to the particularity of the target tracking task, only the label data of the first frame of the picture is provided, and there is a lack of large amounts of data to train the neural network. The general practice

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Target tracking method, system, device and medium based on two-stream convolution neural network
  • Target tracking method, system, device and medium based on two-stream convolution neural network
  • Target tracking method, system, device and medium based on two-stream convolution neural network

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0074] Example 1:

[0075] The three-dimensional convolutional neural network is a type of convolutional neural network. It originated in the fields of motion, limb, and gesture detection. It is different from the two-dimensional convolutional neural network commonly used in image classification and detection. It adds a time dimension. It has excellent time-series feature expression ability and was later introduced into the field of video classification and retrieval.

[0076] Different from tasks such as image classification, the visual target tracking task not only needs to extract the characteristics of the target itself, but also needs to extract the movement change information of the target between video frames, that is, the timing characteristics. The present invention provides a target tracking method based on a dual-stream convolutional neural network. The method applies a three-dimensional convolutional neural network to the field of visual target tracking for the first ti...

Example Embodiment

[0121] Example 2:

[0122] Such as Figure 7 As shown, this embodiment provides a target tracking system based on a dual-stream convolutional neural network. The system includes a first building module 701, a second building module 702, an additive fusion module 703, a third building module 704, and a bounding box. Regression module 705, offline training module 706 and online fine-tuning module 707, the specific functions of each module are as follows:

[0123] The first construction module 701 is used to construct a spatial stream two-dimensional convolutional neural network to extract the feature information of the image block in the current frame. The first construction module 701 is as Figure 8 As shown, specifically including:

[0124] The first input unit 7011 is configured to perform Gaussian sampling of S image blocks in the current frame based on the target neighborhood in the previous frame of the current frame, as the input of the spatial stream two-dimensional convolutio...

Example Embodiment

[0148] Example 3:

[0149] This embodiment provides a computer device. The computer device may be a desktop computer, which includes a processor, a memory, a display, and a network interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory is the operation of the operating system and the computer program in the non-volatile storage medium. An environment is provided, and when the processor executes the computer program stored in the memory, it realizes the target tracking method of the above embodiment 1, as follows:

[0150] Construct a spatial stream two-dimensional convolutional neural network to extract the characteristic information of the image block in the current frame;

[0151] Construct a time serie...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a target tracking method, system, device and medium based on a dual-stream convolution neural network. The method comprises the following steps: a spatial stream two-dimensional convolution neural network is constructed to extract the characteristic information of an image block in a current frame; a three-dimensional convolutional neural network is constructed to extract the motion information of the objects between frames in a video sequence within a certain time range; additive fusion of feature information of spatial flow two-dimensional convolution neural network and timing sequence flow three-dimensional convolution neural network; according to the fused feature information, a fully connected sub-network is constructed to extract the image blocks that meet therequirements. Boundary box regression is used to get the predicted position and size of the object in the current frame. Two-dimensional convolution neural network of spatial flow and three-dimensional convolution neural network of temporal flow are trained off-line before target tracking. In the process of target tracking, the on-line fine tuning of all-connected subnetworks is carried out. Theinvention achieves good tracking effect.

Description

technical field [0001] The invention relates to a target tracking method, in particular to a target tracking method, system, computer equipment and storage medium based on a dual-stream convolutional neural network, belonging to the field of target tracking of computer vision. Background technique [0002] Visual target (single target) tracking task has been a research hotspot in the field of computer vision and has been widely used, especially in recent years with the rapid development of scientific and technological productivity, video surveillance, drone flight, automatic driving and other fields are in urgent need of excellent target tracking algorithm. [0003] The visual target tracking task describes that in a given video sequence scene, only the position of the target in the first frame is provided, and then the next position and size of the target is predicted by the algorithm. Although a large number of algorithms have emerged in recent years, there is still no go...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/20
CPCG06T7/207G06T7/246G06T2207/10016G06T2207/20081G06T2207/20084
Inventor 黄双萍伍思航李豪杰
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap