9 results about "Gesture tracking" patented technology

There is provided techniques for gesture recognition in a gesture tracking application. A method is performed by a controller device. The method comprises receiving user input from a user of the gesture tracking application. The user input identifies a set of gestures for the gesture tracking application. The method comprises training a BSMGRS and a CAMGRS to independently of each other recognize the set of gestures for the user. The method comprises establishing a database of gestures for the user. The database of gestures comprises a set of approved gestures comprises those of the gestures in the set of gestures for which the training of both the BSMGRS and the CAMGRS was successful. The method comprises performing gesture recognition of the user for the gesture tracking application. The gesture recognition is performed using the BSMGRS and in accordance with the established database of gestures.

This invention discloses a high-precision gesture tracking method and system based on hand and controller fusion, belonging to the field of virtual reality and augmented reality technology. It addresses the technical problems in existing VR / AR systems, such as unknown extrinsic parameter estimation for hands and controllers, difficulty in tightly coupling low-frequency vision with high-frequency IMU, and self-occlusion issues in handheld states. The method includes: determining the grip state between the controller and hand based on acquired multi-source data; if the grip state is unoccluded or low-occlusion, performing optimal solution calculations under relevant kinematic constraints to obtain the wrist 6-DoF pose; if the grip state is high-occlusion, performing regression mapping processing on the infrared spot data and controller IMU data based on relevant spatiotemporal feature sequences; dynamically switching the gesture tracking between the controller and hand, and outputting the final hand posture data.

There is provided techniques for switching power modes in a gaze-and-gesture tracking application. A method is performed by a controller device. The method comprises activating a first UI input mode for the gaze-and-gesture tracking application. The first UI input mode is based on a first combination of EOG-based or camera-based eye tracking and EMG-based or camera-based gesture tracking. The method comprises, responsive to detecting that eye tracking and / or gesture recognition performance of the first UI input mode fails to fulfil a performance criterion for the gaze-and-gesture tracking application, activating a second UI input mode for the gaze-and-gesture tracking application. The second UI input mode is based on a second combination of EOG-based or camera-based eye tracking and EMG-based or camera-based gesture tracking, different from the first combination.

The application provides a three-dimensional gesture tracking method based on an RGB camera, which comprises the following steps: taking the RGB camera as an input, performing standardization processing on a picture, and sending the picture into a three-dimensional joint detection module to extract image features, predict a preliminary three-dimensional joint and calculate a hand skeleton length; using a particle swarm optimization algorithm to iteratively fit the hand skeleton length and a MAMO hand model, finding an optimal hand shape and obtaining a new three-dimensional joint; sending the new three-dimensional joint into an inverse kinematics module to solve posture parameters required by the MAMO model; using a MANO model to combine the optimal hand shape and the posture parameters θ to obtain final joints and grid vertices; and performing reconstruction rendering on the obtained grid vertices and joints to output a final three-dimensional hand real-time tracking posture. The method provided by the application has high prediction accuracy and high three-dimensional gesture tracking effect precision, and the tracking process and result are easy to realize.

The application relates to the technical field of medical education, in particular to a central venous catheter medical operation training method and system based on virtual reality and gesture tracking, which comprises the following steps: collecting user hand action data in real time, identifying standard central venous catheter operation actions performed by a user based on the hand action data through a hierarchical fusion gesture recognition algorithm, analyzing the current operation intention of the user in combination with virtual reality visual scene information, generating and outputting corresponding multi-modal tactile feedback signals according to the identified operation actions and the current operation intention, quantitatively evaluating the operation of the user through an evaluation engine based on the hand action data, the operation action recognition result and the current operation intention, and generating a comprehensive evaluation report. Through high-precision gesture tracking, intelligent intention understanding, multi-modal tactile feedback and automatic quantitative evaluation, the application realizes high immersion of the training process and objective and accurate evaluation of operation skills, and effectively improves the authenticity, safety and teaching efficiency of medical operation training.

There is provided techniques for eye tracking and gesture recognition in a gaze-and-gesture tracking application that combines eye tracking for object selection and gesture recognition for action selection. The gaze-and-gesture tracking application is to be run in an XR device that comprises a display. The method is performed by a controller device. The method comprises activating a first UI input mode for the gaze-and-gesture tracking application based on display-based eye tracking. The display-based eye tracking is performed for detecting user selection of objects as rendered on the display. The method comprises, responsive to detecting a critical low power condition, deactivate the display. The method comprises activating a second UI input mode for the gaze-and-gesture tracking application based on display-less eye tracking and / or gesture tracking. The display-less eye tracking and / or gesture tracking is performed for detecting user selection of objects without any visual feedback.

This application provides a method and system for virtual tactile interaction of cultural relics based on multimodal perception fusion. The method includes: preprocessing a 3D ancient scene and a 3D digital model of the cultural relic corresponding to a physical cultural relic model; transmitting one of the physical cultural relic models of different materials to a designated interactive area according to the progress of content preview; identifying the spatial information of the physical cultural relic model using spatial positioning technology, and fusing the spatial information of the physical cultural relic model with the pose of a virtual camera in a VR headset; activating a gesture tracking device to perform real-time gesture tracking, acquiring real-time gesture tracking data, and rendering a virtual hand in the virtual space based on the real-time gesture tracking data; when a viewer wears VR headsets to observe the virtual scene, while the real hand touches the physical cultural relic model, the virtual hand simultaneously touches the virtual cultural relic model in the virtual space. This application improves the accuracy of virtual-real pose fusion during virtual cultural relic interaction, reduces interaction response latency, and provides a better user experience.

The application relates to the technical field of motion capture technology, and discloses a motion capture system for tracking finger postures by fusing light and inertia, which comprises a light-inertia motion capture glove and a light-inertia fusion tracking module. The glove is provided with one handback light-inertia rigid body and ten finger light-inertia single points, and each of the light-inertia single points is integrated with a 6-axis IMU and an active light marker point with a unique flickering code. The tracking module asynchronously runs an IMU thread, a tracking thread and a motion capture thread. After static initialization and light-inertia coordinate system self-calibration, a tight-coupling light-inertia fusion algorithm is used to solve the postures, and 16 segment skeletons with 31 degrees of freedom are output. The application combines the advantages of optical and inertial sensing, has strong anti-shielding capability and no magnetic field interference, suppresses posture drift and zero offset error through double closed-loop feedback, has a short initialization process time, and is suitable for high-precision finger motion capture in scenes such as VR / AR, embodied intelligence and medical rehabilitation.