A joint gesture tracking and recognition strategy for deskVR is proposed

Abstract

The invention discloses a gesture tracking recognition joint strategy method oriented to deskVR, At first, that hand gesture public data set ICVL is train offline by the convolution neural network, get a good robustness, models with high accuracy and fast recognition speed, and by testing the balance point Rate of accuracy and speed of the combined method with synthetic data, Then the depth imagesare captured by the depth camera in real time. After a series of preprocessing, the images are transferred into the gesture recognition method based on the fast model or the five-region convolution neural network model according to the proportion of the hand images, and the 3D information of the recognized joints is returned, and then the 3D human body model is mapped. The recognition method of the invention has good interactive function under the deskVR application environment, and performs well in recognition rate, operation speed, universality, accuracy and the like.

Description

technical field [0001] The invention belongs to the technical field of human-computer interaction, and in particular relates to a joint strategy method for gesture tracking and recognition facing deskVR. Background technique [0002] The poster child for immersive applications is desktop VR. It can be widely used for data analysis, watching VR movies. These applications typically provide stereo vision and head tracking, while they also need to provide gesture recognition for human-computer interaction. Sitting in a chair and resting your elbows on a table enables long gesture interactions in a VR environment. Because head-mounted displays (HMDs) block the eyes from viewing the real world, mice and keyboards are no longer suitable for immersive virtual environments. Joysticks are the most mature solution at the lowest cost, however the user has no direct control. [0003] Currently, gesture sensors on the market, including Leap Motion, use "HMD + gesture sensor" to achiev...

AI Technical Summary

Problems solved by technology

Current gesture recognition methods still struggle to accurately identify joint positions under large occluded regions

3) Easy fatigue: desktop VR experience requires users to wear VR equipment

Traditional model-based methods have faster speed but lower recognition accuracy for complex gestures

The complex CNN network has high precision but poor real-time performance, occupying a lot of GPU computing resources

Images