VR Joint Strength Assessment Using RGB-D Force Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote assessment methods for joint strength, such as sensor-based and motion capture systems, require specialized setups and are not conducive to asynchronous use cases, and existing tele-medicine solutions lack effective non-invasive tracking and feedback mechanisms.
Innovation Solution
A virtual reality system using RGB-D cameras generates a personalized humanoid avatar within a virtual environment, allowing for real-time and asynchronous remote strength assessment by tracking joint angles and estimating force using an inverse dynamics solver, with feedback provided through interactive exercise games.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor-based approaches are used for remote strength assessment, then measurement precision is improved, but device complexity increases and ease of operation deteriorates due to requiring technical experts for sensor placement
Solution Approach 1:
The patent replaces complex mechanical sensor-based measurement systems with a computer vision system using RGB-D cameras. The depth camera captures 3D skeletal data to infer joint positions and movements, eliminating the need for physical sensors attached to the patient's body. This substitution maintains measurement capability while dramatically reducing setup complexity and improving ease of operation.
Solution Approach 2:
The system creates a virtual 3D skeletal model (copy) of the patient's body based on depth camera data. This virtual model replicates the physical body's joint positions and movements, allowing strength assessment without direct physical measurement. The copy enables accurate tracking of joint angles and movements while avoiding the complexity of physical sensor attachment.
2Measurement precision
If motion capture methods are used, then measurement precision is improved, but device complexity increases due to requiring multiple specialized cameras and calibration
Solution Approach 1:
The patent extracts and utilizes only the depth sensing capability from complex motion capture systems. By using a single RGB-D camera's depth stream instead of multiple specialized motion capture cameras, the system achieves joint tracking functionality while eliminating the need for multi-camera calibration and synchronized setup. This extraction approach maintains essential measurement precision while reducing system complexity.
Solution Approach 2:
The RGB-D camera serves multiple functions: it captures color images for visual reference, depth data for 3D skeletal tracking, and provides the basis for inferring joint positions and movements. This multi-functional approach replaces the need for multiple specialized cameras, each performing a single function, thereby reducing device complexity while maintaining measurement precision.
3Loss of information
If haptics-based methods are used, then feedback quality is improved, but device complexity increases and cost increases
Solution Approach 1:
The patent replaces complex haptic feedback devices with a software-based feedback system displayed through the virtual reality interface. The system provides visual feedback through the avatar's movements and graphical representations of joint angles and force estimates, eliminating the need for expensive haptic actuators while maintaining informative feedback quality.
4Ease of operation
If video-chat application is used for remote assessment, then ease of operation is improved, but measurement precision deteriorates due to lack of tracking capability
Solution Approach 1:
The patent merges the simplicity of video-chat based remote assessment with the precision of motion tracking by integrating RGB-D camera depth sensing into the telemedicine platform. The system maintains the accessible remote interaction model while adding automatic 3D skeletal tracking and joint angle measurement capabilities, thereby achieving both ease of operation and measurement precision simultaneously.
Data Source
AI summary
Methods, systems, and apparatuses are described for estimating the force acting on at least one joint of a user while the user engages at least one virtual object within a virtual environment. Motion data associated with joint data of at least one joint of a user may be received from a sensor. The joint data may be used to determine force information. The force information may be used to determine user strength associated with the at least one joint.