Room VR-oriented whole-body three-dimensional posture tracking method

Abstract

The invention discloses a room VR-oriented whole-body three-dimensional posture tracking method. The method comprises the following steps: (1) dividing 17 joints of a human body into five kinematic chains C1, C2, C3, C4 and C5; (2) acquiring the position of each joint on the C2 chain according to a two-arm hinge joint structure by head display position information provided by a VR HMD, and (3) acquiring the postures of C1 and C3 kinematic chains according to a three-arm hinge joint structure by handle position information provided by a VR handle; (4) establishing and training a lower limb kinematics model through C4 and C5 kinematic chains; (5) processing the captured lower limb image by adopting a low-pass filter; (6) transmitting the processed images into a trained model to be recognized, and obtaining 3D position information of nine joint points of the lower limbs; and (7) unifying the obtained upper limb joint positions and lower limb joint positions under the same coordinate system by taking pelvis joints as common root joints. According to the invention, the tracking accuracy and real-time performance can be improved.

Description

technical field [0001] The invention belongs to the technical field of human-computer interaction, and in particular relates to a whole-body three-dimensional posture tracking method for room virtual reality applications. Background technique [0002] A large number of room VR applications require a full-body tracking system, which allows us to see the virtual body in VR, further enhancing the sense of immersion. At present, there are many hardware products for whole-body tracking in the field of room VR exploration. These hardware products allow players to make accurate and real-time interaction and feedback in the virtual world by accurately positioning and tracking the player's whole body. Most VR action games, in order to obtain better interactive feedback, will use handles in the application (the handle can convey interactive feedback to the user through vibration). HTC's inverse kinematics full-body tracking solution, by tracking the headset and two motion controller...

AI Technical Summary

Problems solved by technology

[0004] Most of the existing full-body tracking systems use the positioning function of the head-mounted display and the handle tracker, and use inverse kinematics to solve the upper body motion. However, the human body motion is complex after all, and the elbow positioning is inaccurate through IK, which is very easy to appear. Unnatural movements, while the lower body is often ignored

Final IK can track the lower body through a more complex inverse motion solution, but the ambiguity of the knee position still cannot be resolved

Relying on deep learning to identify the position of human joints is certainly possible, but it causes a waste of existing equipment information. At the same time, serious self-occlusion in some complex human movements reduces the accuracy of deep learning methods.

Images