On-Device Pose Labeling for Personalized Real-Time Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to accurately monitor and adapt to individual user-specific environments and characteristics in real-time pose estimation, leading to reduced accuracy and effectiveness in physical activity performance feedback.
Innovation Solution
A pose monitoring platform that utilizes lightweight models on user devices, combined with periodic updates from more complex models, to personalize and improve pose estimation accuracy by leveraging local data and confidence metrics, while maintaining low computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a complex machine learning model is used for pose estimation, then pose estimation accuracy is improved, but computational requirements and device performance are compromised
Solution Approach 1:
The system segments the pose estimation task into two parts: a lightweight model running on the user's device for real-time inference, and a separate complex model available for periodic updates. This allows the device to maintain low computational requirements while still achieving high accuracy through the lightweight model that is periodically refined by the complex model's insights.
Solution Approach 2:
The system performs preliminary training of the lightweight model using a complex machine learning model before deployment. The complex model pre-processes and prepares training data, which is then used to train the lightweight model that will run on the device. This preliminary action enables the lightweight model to achieve accuracy comparable to the complex model without requiring continuous high computational resources.
2Measurement precision
If manual pose labeling is used for training data, then training data quality is improved, but time consumption and labor requirements increase
Solution Approach 1:
The system implements self-service through automatic pose labeling. The lightweight model automatically generates pose labels for training data by processing images captured during exercise sessions. This eliminates the need for manual annotation while maintaining high data quality, as the model uses its own inference capabilities to create the training labels, thereby saving significant time and labor resources.
Solution Approach 2:
The system uses feedback from the lightweight model's performance during exercise monitoring to automatically generate training data labels. The model's confidence metrics and pose estimates during real-time exercise execution are fed back to create accurate training labels, enabling continuous improvement of the model without manual intervention and reducing time consumption for data preparation.
3Adaptability or versatility
If personalized pose estimation is implemented, then adaptability to individual users is improved, but device complexity increases
Solution Approach 1:
The system applies local quality by training a personalized lightweight model for each user using only that user's exercise data. Instead of using a single complex model that attempts to handle all users, each user gets a customized lightweight model trained on their specific movement patterns, body type, and exercise technique. This keeps the individual model simple while collectively providing high adaptability across different users.
Solution Approach 2:
The system segments the personalization process by maintaining separate lightweight models for each user rather than using a single complex model that handles all users. Each user's personalized model is independently trained on their data and deployed on their device, which simplifies the overall system architecture while enabling high adaptability to individual user characteristics and exercise styles.
4Speed
If real-time pose monitoring is performed, then feedback及时性 is improved, but computational resources are consumed
Solution Approach 1:
The system segments the computational workload by using a lightweight model for real-time pose estimation during exercise sessions, which consumes minimal computational resources and enables immediate feedback. The complex model is used only for periodic training updates in the background, not during real-time execution. This segmentation allows the system to maintain fast feedback responsiveness while keeping computational resource consumption low during active use.
Solution Approach 2:
The system applies partial action by using a lightweight model that provides sufficient accuracy for real-time feedback without requiring the full computational power of a complex model. The lightweight model performs the necessary pose estimation tasks with adequate precision for exercise monitoring, avoiding excessive computational resource consumption while still delivering timely feedback to users during their exercise sessions.
Data Source
AI summary
The systems and methods for improving pose estimation models are disclosed herein. Digital image data of an environment can be obtained and provided to a first machine learning model. A first confidence metric can be computed for the image. The first confidence metric can be compared with a threshold value and provided to a second machine learning model. A second confidence metric can be generated for training of machine learning models for pose estimation. A generic machine learning model can be updated using model parameters from trained local machine learning models.