A whole body 3D pose tracking method for room VR

Abstract

The invention discloses a whole-body three-dimensional pose tracking method for room VR, including: (1) dividing 17 joints of the human body into five kinematic chains C1, C2, C3, C4, and C5; (2) the head provided by the VR HMD Display the position information, obtain the position of each joint on the C2 chain according to the hinge joint structure of the two arms, (3) obtain the position information of the handle provided by the VR handle, and obtain the postures of the C1 and C3 kinematic chains according to the hinge joint structure of the three arms; (4) by C4 and C5 kinematic chains establish and train the lower limb kinematics model; (5) process the captured lower limb image with a low-pass filter; (6) pass the processed image into the trained model for recognition, and obtain 9 lower limbs 3D position information of joint points; (7) Unify the obtained upper limb joint positions and lower limb joint positions into the same coordinate system with the pelvic joint as the common root joint. With the invention, the tracking accuracy and real-time performance can be improved.

Description

technical field [0001] The invention belongs to the technical field of human-computer interaction, and in particular relates to a whole-body three-dimensional posture tracking method for room virtual reality applications. Background technique [0002] A large number of room VR applications require a full-body tracking system, which allows us to see the virtual body in VR, further enhancing the sense of immersion. At present, there are many hardware products for whole-body tracking in the field of room VR exploration. These hardware products allow players to make accurate and real-time interaction and feedback in the virtual world by accurately positioning and tracking the player's whole body. Most VR action games, in order to obtain better interactive feedback, will use handles in the application (the handle can convey interactive feedback to the user through vibration). HTC's inverse kinematics full-body tracking solution, by tracking the headset and two motion controller...

AI Technical Summary

Problems solved by technology

[0004] Most of the existing full-body tracking systems use the positioning function of the head-mounted display and the handle tracker, and use inverse kinematics to solve the upper body motion. However, the human body motion is complex after all, and the elbow positioning is inaccurate through IK, which is very easy to appear. Unnatural movements, while the lower body is often ignored

Final IK can track the lower body through a more complex inverse motion solution, but the ambiguity of the knee position still cannot be resolved

Relying on deep learning to identify the position of human joints is certainly possible, but it causes a waste of existing equipment information. At the same time, serious self-occlusion in some complex human movements reduces the accuracy of deep learning methods.

Images