Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network

Abstract

The invention discloses an egocentric vision in-the-air hand-writing and in-the-air interaction method based on a cascade convolution nerve network. The method comprises steps of S1: obtaining training data; S2: designing a depth convolution nerve network used for hand detection; S3: designing a depth convolution nerve network used for gesture classification and finger tip detection; S4: cascading a first-level network and a second-level network, cutting a region of interest out of a foreground bounding rectangle output by the first-level network so as to obtain a foreground region including a hand, and then using the foreground region as the input of the second-level convolution network for finger tip detection and gesture identification; S5: judging the gesture identification, if it is a single-finger gesture, outputting the finger tip thereof and then carrying out timing sequence smoothing and interpolation between points; and S6: using continuous multi-frame finger tip sampling coordinates to carry out character identification. The invention provides an integral in-the-air hand-writing and in-the-air interaction algorithm, so accurate and robust finger tip detection and gesture classification are achieved, thereby achieving the egocentric vision in-the-air hand-writing and in-the-air interaction.

Description

technical field [0001] The present invention relates to the fields of computer vision and machine learning, in particular to a first-view aerial handwriting and aerial interaction method based on a cascaded convolutional neural network. Background technique [0002] In recent years, with the rise of virtual reality technology (Virtual Reality, VR) and augmented reality technology (Augmenting Reality, AR), the first perspective (EgocentricVision) gesture interaction technology has attracted extensive attention from academia and industry, especially Google Glass, Microsoft Hololens, etc. The emergence of smart wearable devices and virtual reality devices such as Oculus makes it difficult to apply traditional human-computer interaction methods. An algorithm is urgently needed to help devices understand human interaction needs, such as gesture operations. Gesture interaction technology mainly involves two aspects, gesture recognition and key point positioning. The present invent...

AI Technical Summary

Problems solved by technology

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, provide a first-view aerial handwriting and aerial interaction method based on cascaded convolutional neural networks, and solve the problems of fingertip detection and gesture recognition in dynamic video streams , and use gesture recognition as a signal, and fingertip detection and tracking as a sampling point set to realize a set of air handwriting and air interaction solutions from the first perspective

Images