Gesture recognition method based on global-local RGB-D multimodality

An RGB-D, gesture recognition technology, applied in the field of gesture recognition, can solve the problems of no RGB-D feature extraction, the accuracy does not meet the requirements, etc., to achieve the effect of a wide range of application backgrounds and application scenarios

Active Publication Date: 2021-09-21
SUN YAT SEN UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are many shortcomings in the existing technology, the main shortcoming is that the gesture video can be described globally only by the input data of RGB and RGB-D
However, the accuracy rate of the method based on the global description is far from meeting the requirements on the gesture recognition problem, and there is currently no good method for RGB-D feature extraction for gestures.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gesture recognition method based on global-local RGB-D multimodality
  • Gesture recognition method based on global-local RGB-D multimodality
  • Gesture recognition method based on global-local RGB-D multimodality

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0030] Definition of Terms:

[0031] RGB-D: RGB is a commonly used image color representation method. D refers to Depth Image, which is the representation of a depth image. Its format is a picture, and the data content is the value of the distance between the object captured by the camera and the camera. 0,255].

[0032] Such as figure 1 As shown, the present invention is based on the global-local RGB-D multimodal gesture recognition method, mainly for the gesture video input by RGB-D, through the human body bone extraction technology based on RGB-D to the human body and hand bones in the video Estimation, the local data representation of 5 different data modalities (skeleton, RGB map, depth map, RGB optical flow map and depth optical flow map) are respectively constructed through the estimated bones. And by combining with the global data expression of the above modalities, the global-local data of each modality is obtained to calculate the gesture category score, and final...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a gesture recognition method based on global-local RGB-D multimodality. The invention mainly expresses the input gesture video through data modes including bone position, RGB image, depth image and optical flow image, and obtains After the multimodal gesture data is represented, the convolutional neural network and recurrent neural network are used to express the features of the gesture data of different modalities, and the gestures are classified by using the features obtained in different modalities. Finally, the gesture scores of different categories obtained in different modalities are fused to obtain the final gesture classification result based on multimodality. The present invention can be applied to the client or the cloud to recognize the gesture video input by the user, and make the software and hardware of the computer or mobile phone make a corresponding response through the gesture input.

Description

technical field [0001] The present invention relates to the technical field of gesture recognition, in particular to a gesture recognition method based on global-local RGB-D multimodality. Background technique [0002] With the development of science and technology, gesture recognition technology is being used more and more widely. The existing technical inventions mainly obtain gesture videos through RGB cameras or RGB-D cameras. According to the single mode of RGB or the two modes of RGB-D Perform gesture recognition. However, there are many shortcomings in the prior art. The main shortcoming is that the gesture video is described globally only by the input data of RGB and RGB-D. However, the accuracy rate of the method based on the global description is far from meeting the requirements on the gesture recognition problem, and there is currently no good method for RGB-D feature extraction for gestures. Contents of the invention [0003] The main purpose of the present ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/00G06K9/46
CPCG06V40/113G06V10/44
Inventor 郑伟诗李伟宏李本超
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap