Monocular three-dimensional gesture tracking method

Abstract

The invention provides a monocular three-dimensional gesture tracking method. The method comprises the steps of training a hand detection model and a skeleton point recognition model, starting a handdetection model and a tracking module according to the detection number of the hands in the previous frame of image, carrying out skeleton point identification on the region of interest of the currentframe in the Trackhand through a skeleton point identification model, and carrying out smooth filtering processing on the identified skeleton points; counting data about the position and posture of the head in each frame of image; storing the data of the head into a queue Trackhead of the tracking module in real time; determining three-dimensional skeleton coordinates of the skeleton points subjected to smooth filtering processing in combination with head data in Trackhead; rendering the three-dimensional skeleton coordinates to complete gesture tracking. According to the invention, one monocular camera is used for replacing two infrared binocular cameras, the cost is reduced, even if the two monocular cameras are installed, the cost is lower than that of one infrared binocular camera, the overall energy consumption and heat dissipation are reduced, the overall mass of head wearing is reduced, and the comfort of head wearing is improved.

Description

technical field [0001] The present invention relates to the field of computer vision, and more specifically, to a monocular-based three-dimensional gesture tracking method. Background technique [0002] In order to enhance the immersion of VR / AR / MR combined with virtual reality and make VR / AR / MR have a better experience, the human-computer interaction module is essential, especially the 3D posture of the hand in the VR / AR / MR scene. Real-time restoration of precision greatly affects the user's experience of immersion in VR / AR / MR scenes. [0003] At present, in the field of VR / AR / MR, on the mainstream head-mounted all-in-one machine, it is necessary to add an additional gesture recognition tracker. The traditional method is to add two additional infrared binocular cameras or depth cameras to achieve finger tracking, but in In the field of VR / AR / MR, there will be the following problems: 1. Increased additional cost; 2. Increased additional power consumption. Now the mainstream...

AI Technical Summary

Problems solved by technology

[0005] In view of the above problems, the purpose of the present invention is to provide a single-purpose three-dimensional gesture tracking method to solve the problem of high cost, high power consumption, and high heat dissipation of infrared binocular cameras in existing methods, which increase the complexity of the entire head-mounted structure design. Increase the size of the headset, the user will feel uncomfortable wearing it for a long time, and the viewing angle is small, which makes it easy to track the movement trajectory of the hand incompletely

Images