Wearable IMU Exercise Recognition via Sensor Data Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing activity recognition systems using inertial sensors rely on complex feature extraction and image-based models, which require expertise and are not universally applicable across different exercises and users, and are prone to noise and false positives.
Innovation Solution
A method utilizing multiple Inertial Measurement Units (IMUs) in wearable devices like smartwatches and earbuds to generate images from sensor data, employing machine learning models like CNNs for exercise detection, with fixed window segmentation and auto-scaling, allowing users to retrain models for customizable exercise recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex feature extraction methods are used for activity recognition, then measurement precision may be improved, but device complexity increases significantly
Solution Approach 1:
The patent replaces complex mechanical feature extraction and traditional machine learning models with a deep learning neural network that automatically learns features from raw sensor data. The neural network architecture substitutes the manual feature engineering process, allowing the system to achieve high recognition accuracy without requiring complex preprocessing steps or domain expertise in selecting appropriate features.
Solution Approach 2:
The neural network performs self-service by automatically learning and extracting relevant features from raw sensor data during training and inference. Instead of requiring external expertise to design and tune feature extraction algorithms, the model autonomously adapts to different exercise types and user patterns, reducing the need for manual intervention and complex system configuration.
2Measurement precision
If custom analytic window detection methodology is developed for each exercise type, then measurement precision improves, but device complexity and ease of operation deteriorate
Solution Approach 1:
The patent implements a universal neural network model that can detect and classify multiple exercise types using a single unified architecture. The model processes raw sensor data from various exercises (squats, pushups, pullups, etc.) through the same computational layers, eliminating the need for exercise-specific window detection methodologies while maintaining high accuracy across all exercise types.
Solution Approach 2:
The neural network dynamically adapts its internal parameters (weights and biases) based on the specific exercise being performed, allowing it to accurately detect different exercise types without changing its structural architecture. This parameter adaptation enables the system to handle diverse exercise patterns while maintaining a simple, unified detection framework.
3Ease of operation
If threshold-based peak detection is used for windowing, then ease of operation improves, but reliability deteriorates due to noise and user variation
Solution Approach 1:
The patent replaces fragile threshold-based peak detection with a robust neural network-based approach. Instead of using fixed thresholds that fail under noise and user variation, the neural network learns adaptive decision boundaries during training, making the system resilient to noise, different user intensities, and varying exercise execution styles while maintaining operational simplicity.
Solution Approach 2:
The neural network incorporates feedback mechanisms through its training process, where it continuously adjusts its parameters based on labeled exercise data. This feedback loop enables the model to learn optimal detection strategies that account for noise and user variation, improving reliability without complicating the operational interface.
4Measurement precision
If extensive feature engineering is performed, then measurement precision may improve, but loss of time increases due to complex processing
Solution Approach 1:
The patent substitutes time-consuming manual feature engineering with efficient neural network-based automatic feature learning. The model processes raw sensor data directly through optimized computational layers, eliminating the need for multiple sequential feature extraction steps and reducing overall processing time while maintaining or improving classification accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides for using multiple inertial measurement units (IMUs) to recognize particular user activity, such as particular types of exercises and repetitions of such exercises. The IMUs may be located in consumer products, such as smartwatches and earbuds. Each IMU may include an accelerometer and a gyroscope, each with three axes of measurement, for a total of 12 raw measurement streams. A training image includes a plurality of subplots or tiles, each depicting a separate data stream. The training image is then used to train a machine learning model to recognize IMU data as corresponding to a particular type of exercise.