Dynamic gesture recognition method and system based on deep neural network

A deep neural network, dynamic gesture technology, applied in the field of computer vision and pattern recognition, can solve the problem of low recognition rate

Active Publication Date: 2018-12-04
广州智能装备研究院有限公司
View PDF8 Cites 64 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The technical problem to be solved by the present invention is the problem that the recognition rate of the existing dynamic gesture recognition method is not high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic gesture recognition method and system based on deep neural network
  • Dynamic gesture recognition method and system based on deep neural network
  • Dynamic gesture recognition method and system based on deep neural network

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment 1

[0074] Such as figure 1 As shown, the dynamic gesture recognition method based on deep neural network provided by the present invention comprises the following steps:

[0075] Step 10: Acquire dynamic gesture data samples, this step includes:

[0076] Step 11: Use the 3D depth camera to collect C kinds of dynamic gesture video clips with different meanings, and collect at least 50 different video clips for each gesture.

[0077] Sampling n frames of RGB images and corresponding depth information images at equal intervals for each dynamic gesture video segment to form a sample x i ={x i1 ,x i2 ,...,x ik ,...,x in}, where x ik for sample x i The k-th frame data in is a four-channel data in RGB-D format with a size of 640×320×4, and C is a positive integer;

[0078] Step 12: Label all the collected video clips with gesture information, and each video corresponds to a unique gesture label, which is used as a training sample data set.

[0079] Among them, sample x i Form ...

specific Embodiment 2

[0091] For the training of the neural network model, the number of samples is of great significance to the training results. In order to reduce the workload of sample collection, the present invention proposes methods such as random translation, flipping, noise addition, and deformation of each video in the training sample data set. Carry out expansion, and form the final training sample data set with the expanded training samples and the original training samples to form a training sample library.

[0092] For each sample x in the training sample data set i The method of performing translation operation is as follows:

[0093] will sample x i The coordinates (x, y) of any pixel point on each channel in each frame of RGB-D data are translated along the x-axis by t x units, translate t along the y-axis y units, get (x',y'), where x'=x+t x , y'=y+t y , t x with t y They are random integers between [-0.1×width, 0.1×width] and [-0.1×height, 0.1×height], and the width is x ...

specific Embodiment 3

[0104] This specific embodiment 3 is a further refinement of the dynamic gesture recognition network model based on the deep neural network designed in the specific embodiment, and the specific steps include:

[0105] Step 21: The method of designing the feature extraction network is as follows:

[0106] Using a 4-layer convolutional neural network for a video input sample x of gesture meaning i The four-channel data in RGB-D format of n frames (n is a positive integer) with a size of 640×320×4 is used for feature extraction, and the convolution kernels of the first to fourth convolutional layers are set to 32, 64, 128, 256.

[0107] Then, in each convolutional layer, the convolution kernel window size is set to 3×3, and the window sliding step is set to 2; the maximum pooling window is set to 2×2, and the window sliding step is set to 2; the final output n features of size 2×1×256.

[0108] The n 2×1×256 features of the final output are pulled into a column vector to form ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a dynamic gesture recognition method and system based on a deep neural network. The dynamic gesture recognition method comprises the steps of collecting dynamic gesture video clips with different gesture meanings to generate a training sample data set, wherein the sample data includes RGB images and depth information; designing a dynamic gesture recognition network model based on the deep neural network, and training the model by using the training samples; and performing dynamic gesture testing and recognition by using the trained dynamic gesture recognition model. Thedynamic gesture recognition network model is composed of a feature extraction network, a front and back frame association network and a classification recognition network, wherein the front and backframe association network is used for performing front and back time frame association mapping on feature vectors obtained through the feature extraction network of the samples of each gesture meaningand merging the feature vectors into a fusion feature vector of the gesture meaning. According to the invention, a bidirectional LSTM model is introduced into the network model to understand the correlation between continuous gesture postures, thereby greatly improving the recognition rate of dynamic gestures.

Description

technical field [0001] The invention relates to the technical field of computer vision and pattern recognition, in particular to a dynamic gesture recognition method and system based on a deep neural network. Background technique [0002] With the rapid development of human-computer interaction technology, the gesture recognition technology, which uses human hand posture as the direct input means of human-computer interaction, is becoming more and more mature. This gesture recognition technology, which uses computers to recognize and judge the meaning of gestures, has a large number of applications in the fields of smart home, smart wear, and augmented reality. [0003] The key technologies of gesture recognition are gesture tracking and gesture recognition. At present, there are two main methods: one is gesture recognition based on static pictures obtained by ordinary cameras. This method uses traditional pattern recognition methods to extract artificial features from gest...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/08G06N3/04
CPCG06N3/084G06V40/28G06N3/045G06F18/214
Inventor 肖定坤万磊詹羽荣李博
Owner 广州智能装备研究院有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap