Regional convolutional neural network-based method for gesture identification and interaction under egocentric vision

Abstract

The invention discloses a regional convolutional neural network-based method for gesture identification and interaction under egocentric vision. The method comprises the following steps of S1, obtaining training data; S2, designing a regional neural network which is used for gesture classification and fingertip detection while being used for hand detection, and ensuring that an input of the neural network is a three-channel RGB image and outputs of the neural network are top left corner coordinates and top right corner coordinates of an external connection matrix of a gesture region, gesture types and gesture skeleton key points; and S3, judging the gesture types, and outputting corresponding interactive results according to different interactive demands. The invention provides a complete method for gesture identification and interaction under egocentric vision. Through single model training and partial network sharing, the identification speed and accuracy of the gesture identification under the egocentric vision are increased and improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision and machine learning, and in particular relates to a first-view gesture recognition and interaction method based on a regional convolutional neural network. Background technique [0002] With the introduction of many virtual reality (Virtual Reality, VR) and augmented reality (Augmenting Reality, AR) equipment products, the enthusiasm for research on human-human interaction has increased a lot. Looking at the whole body of the human body, the hand has been considered as the most important and common interaction tool. At the same time, in the field of human-computer interaction, gesture recognition has become a very important research direction. Due to the rise of AR and VR, Egocentric Vision gesture interaction technology has gradually attracted everyone's attention, and more and more scholars and companies have invested manpower and material resources in corresponding research and develo...

AI Technical Summary

Problems solved by technology

These traditional method models usually require artificially predefined features, such as scale-invariant feature transformation (SIFT), Chen features (SURF), histogram of oriented gradients (HOG), Fourier descriptors and other hand-shaped feature descriptions and based on optical flow, The gesture motion information description of the motion tracking method, the method of manual selection of features has great limitations, usually requires prior knowledge, experience and a lot of manual adjustments, and the recognition rate of the algorithm model is easily affected by the gesture operation speed, direction, and hand size. difference has a big impact on

Images