Skeleton Model Control via Neural Network Orientation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing input devices for virtual and augmented reality applications require a large number of inertial measurement units (IMUs) to track user movements accurately, which can be cumbersome and costly, and suffer from drift errors and accumulated errors over time.
Innovation Solution
The use of a reduced number of micro-electromechanical system (MEMS) IMUs in conjunction with an artificial neural network to predict the orientations of untracked parts of the user, allowing for the construction of a skeleton model that can control computer systems without the need for additional sensor modules on the torso and forearms, leveraging both inertial and optical tracking systems for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of inertial measurement units are used to track user movements accurately, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses optical tracking data to create a virtual copy or representation of the inertial tracking data for untracked body parts. The optical system captures the actual movement, and this captured movement is used to generate predicted orientation data that copies the behavior of what the IMU would have measured, eliminating the need for physical IMU sensors on every body part.
Solution Approach 2:
The patent replaces the mechanical inertial sensing system with an optical tracking system for predicting orientations of untracked body parts. Instead of using physical accelerometers and gyroscopes (mechanical/inertial system) on every body part, the system uses optical cameras to track markers and computationally derive the orientation information that would otherwise require inertial sensors.
2Duration of action of stationary object
If inertial measurement units are used over extended periods, then continuous tracking is achieved, but drift errors and accumulated errors increase
Solution Approach 1:
The patent implements a feedback mechanism where the optical tracking system continuously monitors the actual positions and orientations of tracked body parts, and this feedback information is used to correct the predicted orientations of untracked body parts. The system compares the predicted state with the actually observed state and adjusts the predictions to minimize errors, preventing drift accumulation over time.
Solution Approach 2:
The patent creates a composite tracking system that combines optical tracking data with inertial measurement data. The final orientation estimate is a composite result that leverages the strengths of both systems: the optical system provides absolute position references to prevent drift, while the inertial system provides continuous orientation data. This composite approach maintains accuracy over extended tracking durations.
3Measurement precision
If sensor modules are placed on all body parts including torso and forearms, then complete coverage is achieved, but ease of operation and user comfort decrease
Solution Approach 1:
The patent extracts or removes the requirement for IMU sensors from certain body parts (specifically the torso and forearms) by using the optical tracking system to infer the orientations of these untracked parts from the movements of tracked parts. This extraction eliminates the need for uncomfortable sensors on these body parts while maintaining complete tracking coverage through computational prediction.
Data Source
AI summary
A system having sensor modules and a computing device. Each sensor module has an inertial measurement unit attached to a portion of a user to generate motion data identifying a sequence of orientations of the portion. The computing device provides the sequences of orientations measured by the sensor modules as input to an artificial neural network, obtains as output from the artificial neural network a predicted orientation measurement of a part of the user, and controls an application by setting an orientation of a rigid part of a skeleton model of the user according to the predicted orientation measurement. The artificial neural network can be trained to predict orientations measured using an optical tracking system based on orientations measured using inertial measurement units and/or to prediction orientation measurements of some rigid parts in a kinematic chain based on orientation measurements of other rigid parts in the kinematic chain.