Global-local RGB-D multimode-based gesture recognition method

An RGB-D, gesture recognition technology, applied in the field of gesture recognition, can solve the problem of no RGB-D feature extraction, the accuracy does not meet the requirements, etc., to achieve the effect of improving gesture recognition performance, wide application background and application scenarios

Active Publication Date: 2018-08-10
SUN YAT SEN UNIV
View PDF4 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are many shortcomings in the existing technology, the main shortcoming is that the gesture video can be described globally only by the input data of RGB and RGB-D
However, the accuracy...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Global-local RGB-D multimode-based gesture recognition method
  • Global-local RGB-D multimode-based gesture recognition method
  • Global-local RGB-D multimode-based gesture recognition method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0030] Definition of Terms:

[0031] RGB-D: RGB is a commonly used image color representation method. D refers to Depth Image, which is the representation of a depth image. Its format is a picture, and the data content is the value of the distance between the object captured by the camera and the camera. 0,255].

[0032] Such as figure 1 As shown, the present invention is based on the global-local RGB-D multimodal gesture recognition method, mainly for the gesture video input by RGB-D, through the human body bone extraction technology based on RGB-D to the human body and hand bones in the video Estimation, the local data representation of 5 different data modalities (skeleton, RGB map, depth map, RGB optical flow map and depth optical flow map) are respectively constructed through the estimated bones. And by combining with the global data expression of the above modalities, the global-local data of each modality is obtained to calculate the gesture category score, and final...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a global-local RGB-D multimode-based gesture recognition method. An inputted gesture video is expressed mainly through data modes comprising a bone position, an RGB image, a depth image and an optical flow image; after the multimode gesture data expression is obtained, a method of a convolution neural network and a recurrent neural network is used to carrying out feature expression on gesture data in different modes, and the features obtained in different modes are used for gesture classification; and finally, different classes of gesture scores obtained in different modes are fused to obtain a multimode-based gesture classification result finally. The method can be applied to a client or a cloud for recognizing a gesture video inputted by the user, and through gesture input, a computer or mobile phone software or hardware makes a corresponding response.

Description

technical field [0001] The present invention relates to the technical field of gesture recognition, in particular to a gesture recognition method based on global-local RGB-D multimodality. Background technique [0002] With the development of science and technology, gesture recognition technology is being used more and more widely. The existing technical inventions mainly obtain gesture videos through RGB cameras or RGB-D cameras. According to the single mode of RGB or the two modes of RGB-D Perform gesture recognition. However, there are many shortcomings in the prior art. The main shortcoming is that the gesture video is described globally only by the input data of RGB and RGB-D. However, the accuracy rate of the method based on the global description is far from meeting the requirements on the gesture recognition problem, and there is currently no good method for RGB-D feature extraction for gestures. Contents of the invention [0003] The main purpose of the present ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/46
CPCG06V40/113G06V10/44
Inventor 郑伟诗李伟宏李本超
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap