Unlock instant, AI-driven research and patent intelligence for your innovation.

Gesture action recognition interaction system and method based on hybrid neural network

A hybrid neural network and gesture technology, applied in the field of image recognition, can solve problems such as delay, and achieve the effect of improving accuracy, obvious recognition effect, and improved effect

Active Publication Date: 2021-06-29
NORTH CHINA UNIVERSITY OF TECHNOLOGY
View PDF11 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In real systems for human-computer interaction, automatic detection and classification of dynamic gestures is challenging because (1) people vary greatly in making gestures, recognizing and classifying them; (2) the system must work online to Avoid noticeable delays between performing a gesture and categorizing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gesture action recognition interaction system and method based on hybrid neural network
  • Gesture action recognition interaction system and method based on hybrid neural network
  • Gesture action recognition interaction system and method based on hybrid neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The technical solutions provided by the present invention will be described in detail below in conjunction with specific examples. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0040] The hybrid neural network-based gesture recognition interaction method provided by the present invention, as shown in the figure, includes the following steps:

[0041] Step 1, Image and Video Dataset Acquisition

[0042] Compared with the RGB video used in most public gesture datasets, the depth image and IR video can obtain the depth information of the hand information. In this paper, the TOF depth camera is used to collect the hand data. The collection steps are as follows: figure 2 As shown, it specifically includes the following steps:

[0043] 1) Use the depth camera to shoot 10 segments of depth video, color video, and infrared video in each gestu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a projection gesture action recognition interaction method and system based on a 3D CNN and RNN hybrid neural network, and the method comprises the steps: obtaining a depth image video, a color image video and an infrared image video of a hand through a depth camera, carrying out the format unification of the videos, then grouping and sending video files to a 3DCNN (three-dimensional convolutional neural network) network for video action learning, then outputing image features, then requiring to use an RNN (recurrent neural network) network for loop training, and finally outputing an identification result. According to the method, depth information of hand information can be obtained, the recognition accuracy can be improved, the most advanced performance is achieved on the data set built by the user, and by combining the 3DCNN and RNN hybrid neural network, the fusion effect is greatly improved compared with the previous CNN + RNN algorithm effect.

Description

technical field [0001] The invention belongs to the technical field of image recognition, and relates to a gesture recognition interaction system and method based on a hybrid neural network. Background technique [0002] In recent years, with the rise of artificial intelligence, machine learning and deep learning have rolled up the wave of computers. Human-computer interaction has become the top priority of research in the field of machine vision today. Smart devices with human-computer interaction functions are developing rapidly in the market. Gesture, as the most commonly used human body interaction method in people's daily life, has been applied to many smart devices. [0003] Gestures and gestures are a common form of human communication. Therefore, it is also natural for humans to use this form of communication to interact with machines. For example, simple interactive human-computer interaction can improve the comfort and safety of cars; simple gesture interaction...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06K9/46G06N3/04G06N3/08
CPCG06N3/08G06V40/107G06V40/28G06V10/40G06N3/045
Inventor 王立军于霄洋李争平
Owner NORTH CHINA UNIVERSITY OF TECHNOLOGY