Gesture tracking method for VR headset device and VR headset device

Abstract

The invention provides a gesture tracking method for a VR headset device, which comprises the following steps of: acquiring a plurality of training images; separating hand depth images; marking a three-dimensional gesture, and forming original point cloud; calculating a normal vector and a curvature, and carrying out mean removing normalization; setting up a CNN network, wherein an input end of the CNN network is used for respectively inputting multiple normal vectors, curvatures and hand depth images, and an output end of the CNN network is used for outputting three-dimensional coordinates of a plurality of joint points including a palm center; using the trained CNN network as a feature extractor of the three-dimensional gesture, acquiring real-time action depth images by a depth camera, extracting and processing normal vector, curvature and hand depth image information of the three-dimensional gesture which the real-time action depth images comprise by the feature extractor, outputting the three-dimensional coordinates of a plurality of joint points including the palm center, and carrying out tracking on the identified three-dimensional gesture. The invention further discloses a VR headset device. The gesture tracking method and the VR headset device which are provided by the invention fuse three-dimensional feature information, and have the advantage of high model identification rate.

Description

technical field [0001] The present invention relates to the technical field of virtual reality, in particular to a gesture tracking method for a VR head-mounted device and the VR head-mounted device. Background technique [0002] Virtual reality technology is an important branch direction of simulation technology. Virtual reality technology uses computers, related technologies and software and hardware tools to generate real-time dynamic, three-dimensional and textured images and scenes, and enables it to imitate various human perceptions and interact with users using sensors. Since the emergence of virtual reality technology in 1963, the theory of virtual reality technology has been relatively perfect, and in recent years, virtual reality technology has been widely and deeply researched and used in military simulation, entertainment games, medical treatment, construction and other industries. [0003] In the human-computer interaction process of the existing virtual realit...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem that the three-dimensional structure of the hand has not been effectively utilized during the image acquisition of the existing VR head-mounted equipment, resulting in large data errors and inaccurate tracking gesture coordinates, the present invention discloses a gesture for VR head-mounted equipment tracking method

Images