Human behavior recognition method integrating space-time dual-network flow and attention mechanism

A dual network and recognition method technology, applied in the field of computer vision behavior recognition, to achieve the effect of enhancing motion recognition, improving the overall judgment accuracy, and reducing the amount of calculation

Active Publication Date: 2018-01-19
NANJING UNIV OF POSTS & TELECOMM
View PDF4 Cites 168 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the simple introduction of the attention mechanism fails to effectively identify the

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Human behavior recognition method integrating space-time dual-network flow and attention mechanism
  • Human behavior recognition method integrating space-time dual-network flow and attention mechanism
  • Human behavior recognition method integrating space-time dual-network flow and attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] Below in conjunction with accompanying drawing, technical scheme of the present invention is described in further detail:

[0029] The idea of ​​the present invention is to integrate the spatio-temporal dual network flow and the attention mechanism strategy. First, use the Lucas-Kanade optical flow method from coarse to fine strategy to extract the motion optical flow features in the RGB image video frame, and use the Munsell color conversion system to convert them Convert and generate the optical flow feature image of the corresponding frame to increase the corresponding motion information; then, based on the convolutional neural network (CNN) and the long short-term memory (LSTM) neural network, an independent time flow and space flow network are respectively constructed, and a continuous period of time is selected In the video window, the spatial flow and time flow GoogLenet convolutional neural network model of the corresponding parameters is obtained by transfer lea...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a human behavior recognition method integrating the space-time dual-network flow and an attention mechanism. The method includes the steps of extracting moving optical flow features and generating an optical flow feature image; constructing independent time flow and spatial flow networks to generate two segments of high-level semantic feature sequences with a significant structural property; decoding the high-level semantic feature sequence of the time flow, outputting a time flow visual feature descriptor, outputting an attention saliency feature sequence, and meanwhile outputting a spatial flow visual feature descriptor and the label probability distribution of each frame of a video window; calculating an attention confidence scoring coefficient per frame time dimension, weighting the label probability distribution of each frame of the video window of the spatial flow, and selecting a key frame of the video window; and using a softmax classifier decision to recognize the human behavior action category of the video window. Compared with the prior art, the method of the invention can effectively focus on the key frame of the appearance image in the originalvideo, and at the same time, can select and obtain the spatial saliency region features of the key frame with high recognition accuracy.

Description

technical field [0001] The invention relates to an early human behavior recognition method, in particular to a human behavior recognition method that integrates a spatio-temporal dual network flow and an attention mechanism, and belongs to the technical field of computer vision behavior recognition. Background technique [0002] Human behavior recognition in video sequences is a research topic involving computer vision, pattern recognition and artificial intelligence. It has been a hot research topic because of its wide application value in commercial, medical and military fields. However, due to the diversity and non-rigidity of human behavior and the inherent complexity of video images, it is still a challenging task to propose a robust and real-time accurate method. [0003] At present, most of the research starts from the three aspects of moving target detection, action feature extraction and action feature understanding. The general human action recognition method basic...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06N3/08G06N3/04
Inventor 刘天亮谯庆伟戴修斌刘峰
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap