Activity Label Generation Using Body Graphs for HAR Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of generating reliable and accurate labels for training machine-learning models in Human Activity Recognition (HAR) is significant due to the time-consuming, costly, and error-prone nature of manual annotation, which affects the performance of these models.
Innovation Solution
A method involving the generation of a time-resolved reduced graph representation of human body or body parts from video data, using machine-learning models to associate labels with activities, and combining this with sensor data to create input-output pairs for training datasets, which can be used to train HAR models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual annotation is used to generate labels, then label quality can be ensured, but the process becomes time-consuming and costly
Solution Approach 1:
The system uses automated machine-learning models to generate labels for training data, enabling the system to self-service the label generation process without requiring manual human annotation. This reduces time consumption and costs while maintaining acceptable label quality through automated activity recognition from video and sensor data.
2Measurement precision
If manual annotation is used to generate labels, then accurate labels can be obtained, but errors and costs increase
Solution Approach 1:
The patent replaces the mechanical process of manual human annotation with an automated computational system that uses machine-learning models to generate labels. This substitution reduces costs and eliminates human errors while maintaining label accuracy through automated analysis of video and sensor data.
3Adaptability or versatility
If new data is collected for diverse HAR use cases, then model customization is improved, but the complexity of data collection and labeling increases
Solution Approach 1:
The system employs a universal automated label generation approach that can handle diverse HAR use cases (fall detection, gesture detection, activity classification) using the same underlying technology. The machine-learning models are trained on multi-modal data (video, accelerometer, gyroscope) to create versatile models that can be adapted to different applications without requiring separate manual annotation processes for each use case.
Data Source
AI summary
In accordance with an embodiment, a method, includes: obtaining video data depicting a human body or body part during a time interval; based on the video data, generating a time-resolved reduced graph representation of the human body or body part during the time interval; generating, using one or more machine-learning models operating based on the time-resolved reduced graph representation, a label associated with an activity of the human body or body part during the time interval; obtaining a sensor data observing the human body or body part during the time interval; and storing, in a training dataset for training a further machine-learning model, an input-output data pair comprising the sensor data as input and the label as output.