A method and apparatus for human behavior recognition based on multi-mode channel feature fusion

A feature fusion and human body technology, applied in character and pattern recognition, instruments, calculations, etc., can solve the problems of missing details, difficulty in describing the human body's spatial structure, etc., and achieve the effect of improving the accuracy rate

Active Publication Date: 2018-12-25
深圳市感动智能科技有限公司 +1
View PDF9 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are inherent defects in a single model channel. The depth image is difficult to describe the spatial structure of the human body, and the skeleton image only loses a lot of details about the node position information.
This makes it challenging to extract more descriptive and discriminative behavioral descriptors from a single data model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and apparatus for human behavior recognition based on multi-mode channel feature fusion
  • A method and apparatus for human behavior recognition based on multi-mode channel feature fusion
  • A method and apparatus for human behavior recognition based on multi-mode channel feature fusion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further described below through specific embodiments and accompanying drawings.

[0025] figure 1 It is a flow chart of a human behavior recognition method based on multimodal feature fusion, including the following steps:

[0026] Step 1: Extract AH-DMMs features

[0027] The first step: Given a human body behavior video sequence S=[I 1 , I 2 ,..., I i ,...,I N ], I i Denotes the i-th frame depth image. In order to make full use of the depth information, the depth value of each frame is projected in three orthogonal directions:

[0028] I i →{map f , map s , map t}

[0029] Get the orthographic projection map f , side projection map s and top view projection map t . Then in these three directions, compare the depth maps between t and t-1 frames to find out their motion areas, and then accumulate them in time order to form motion features in three directions . The mathematical expression is as follows:

[0030]

[0031...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a method and a device for recognizing human behavior based on multi-mode channel feature fusion. The method comprises the following steps: 1) constructing an adaptive hierarchical structure for an input depth image sequence; 2) extracting DMM features at each level of the adaptive hierarchical structure; 3) serially connecting DMM features of each level to construct adaptive depth motion map features as depth image channel features; 4) selecting relatively stable skeleton joints as reference joints according to the input skeleton image sequence; 5) calculating the displacement difference between the other joint nodes and the reference joint nodes in each frame as the feature expression in each frame; 6) combining the whole skeleton image sequence to obtain a feature expression of an action sequence as a skeleton image channel feature; 7) through feature fusion and classifying the fused feature, the human body behavior recognition result is obtained. The invention can describe the human body time sequence motion information and the spatial structure information at the same time, and has good recognition effect and robustness.

Description

technical field [0001] The present invention relates to a human body behavior recognition method and device based on multimodal feature fusion. First, a brand new descriptor is proposed from the depth sequence modulus: adaptive depth motion map, which is used to describe the time series motion information of the human body. Then, based on the skeleton sequence modulus, a joint point displacement descriptor is proposed to describe the spatial structure information of the human body. Finally, two fusion strategies are adopted: the decision-level fusion method and the feature-level fusion method, which fuse two feature descriptors for human behavior recognition. Background technique [0002] Human behavior recognition is an important research direction in the field of computer vision, and has important theoretical research significance. The purpose of the research is to automatically analyze the ongoing activity of a sequence of consecutive images in an unknown video. Human b...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/46G06K9/62
CPCG06V40/20G06V10/44G06F18/24
Inventor 丁润伟何侵嵚金永庆刘宏
Owner 深圳市感动智能科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap