Human skeleton behavior recognition method and device based on deep reinforcement learning

A technology of reinforcement learning and human skeleton, applied in reinforcement learning, deep learning, and computer vision fields, it can solve problems such as neglect, and achieve the effect of reducing the amount of calculation, strengthening the discrimination, and improving the performance.

Active Publication Date: 2018-07-20
TSINGHUA UNIV
View PDF2 Cites 72 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most CNN-based methods consider all frames to be equally important, thus ignoring the most critical frames in the video

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Human skeleton behavior recognition method and device based on deep reinforcement learning
  • Human skeleton behavior recognition method and device based on deep reinforcement learning
  • Human skeleton behavior recognition method and device based on deep reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0037]Before introducing the human skeleton behavior recognition method and device based on deep reinforcement learning, let’s briefly introduce deep reinforcement learning and behavior recognition about human skeleton.

[0038] Regarding the behavior recognition task of the human skeleton, there are more than 40 public data sets that can be used for experimental training and testing, among which the more mainstream ones are NTU-RGBD, SYSU-3D, UT-Kinect, etc. NTU-RGBD is currently the largest dataset, containing 56,880 videos cap...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a human skeleton behavior recognition method and device based on deep reinforcement learning. The method comprises: uniform sampling is carried out on each video segment in a training set to obtain a video with a fixed frame number, thereby training a graphic convolutional neural network; after parameter fixation of the graphic convolutional neural network, an extraction frame network is trained by using the graphic convolutional neural network to obtain a representative frame meeting a preset condition; the graphic convolutional neural network is updated by using the representative frame meeting the preset condition; a target video is obtained and uniform sampling is carried out on the target video, so that a frame obtained by sampling is sent to the extraction frame network to obtain a key frame; and the key frame is sent to the updated graphic convolutional neural network to obtain a final type of the behavior. Therefore, the discriminability of the selectedframe is enhanced; redundant information is removed; the recognition performance is improved; and the calculation amount at the test phase is reduced. Besides, with full utilization of the topologicalrelationship of the human skeletons, the performance of the behavior recognition is improved.

Description

technical field [0001] The invention relates to the technical fields of computer vision, reinforcement learning and deep learning, in particular to a human skeleton behavior recognition method and device based on deep reinforcement learning. Background technique [0002] Action recognition aims to distinguish the action category in a given video, and is an important research direction in computer vision. Behavior recognition has a wide range of applications, such as video surveillance, human-robot interaction, etc. Compared with the traditional color video, the skeleton-based video contains the 3D position of the key bones of the human body, and it has higher robustness to the transformation of the viewing angle, the scale of the human body and the speed of motion. Moreover, with the development of depth sensors (such as Kinect) and the maturity of human pose estimation algorithms, data based on human skeletons is increasing day by day. Therefore, the research on behavior ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04
CPCG06N3/04G06V40/23G06V20/42G06F18/214
Inventor 鲁继文周杰唐彦嵩田毅
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap