Behavior recognition method for learning human skeleton of neural network based on end-to-end space-time diagram

A neural network and human skeleton technology, applied in the field of computer vision, can solve problems such as not considering semantic information

Active Publication Date: 2019-06-07
ZHEJIANG UNIV
View PDF4 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Traditional methods often regard the human skeleton as a static invariant graph structur...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Behavior recognition method for learning human skeleton of neural network based on end-to-end space-time diagram
  • Behavior recognition method for learning human skeleton of neural network based on end-to-end space-time diagram
  • Behavior recognition method for learning human skeleton of neural network based on end-to-end space-time diagram

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0105] The implementation method of this embodiment is as described above, and the specific steps will not be described in detail. The following only shows the effect of the case data. The present invention is implemented on two data sets with ground-truth labels, namely:

[0106] NTU-RGB+D dataset: This dataset contains 37920 training skeleton sequences and 18960 test skeleton sequences;

[0107] Kinetics dataset: This dataset extracts the 2D skeleton sequences in the Kinetics video dataset, including 240,000 training skeleton sequences and 20,000 test skeleton sequences;

[0108] The main process of skeleton-based behavior recognition is as follows:

[0109] 1) Using the results of each frame clustering to obtain the spatial node relationship of the skeleton sequence;

[0110] 2) Use the trajectory of each node to obtain the time node relationship of the skeleton sequence;

[0111] 3) Use a 10-layer graph convolutional network, where the graph input of each layer of the g...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a behavior recognition method for learning a human skeleton of a neural network based on an end-to-end space-time diagram. The method is used for behavior recognition of a human 3D skeleton. The method specifically comprises the following steps: obtaining a human body 3D skeleton key point position data set for training, and defining an algorithm target; performing clustering expression on each frame based on the spatial position to obtain a spatial node relation; calculating a time track of each joint point, and performing relation measurement according to the time track to obtain a time node relation; establishing a joint learning framework of the space-time diagram learning and the diagram convolutional neural network; and estimating the behavior category of thecontinuous human body 3D skeleton by using the learning framework. The method is suitable for human body action analysis in a real video, and has a good effect and robustness for various complex conditions.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular relates to an end-to-end spatio-temporal graph learning neural network behavior recognition and detection method for a human body 3D skeleton. Background technique [0002] The problem of behavior recognition based on human skeleton is defined as the following problem: in a sequence of human skeleton key point positions containing multiple frames, predict the behavior category. Human skeleton joints are often used as auxiliary information for some high-level visual tasks, such as video abnormal behavior detection, video action recognition, etc. The key factors of human skeleton behavior recognition include modeling of temporal structure, correspondence between joint points and computational efficiency. Traditional methods often regard the human skeleton as a static invariant graph structure, without considering the semantic information associated between nodes under specific acti...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/62G06N3/04
Inventor 李玺李斌张仲非
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap