Action Interval Estimation Model With Augmented HSMM Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost of generating supervised data for training models to estimate time segments of actions is a significant challenge.
Innovation Solution
An action segment estimation model is built using a hidden semi-Markov model, where observation probabilities are learned through unsupervised learning, and transition probabilities are learned using supervised learning with augmented supervised data generated by oversampling in time and feature space, incorporating teacher information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised data is generated manually for training action segment estimation models, then model training accuracy is improved, but data generation cost and time consumption increase significantly
Solution Approach 1:
The system performs preliminary unsupervised learning to extract movement patterns and templates before the actual supervised training. By pre-processing the data to identify and categorize movement types, the system reduces the manual annotation burden while maintaining training accuracy. The movement templates generated in advance serve as reusable components for subsequent supervised learning.
Solution Approach 2:
The system creates synthetic training data by copying and transforming existing movement templates. Through data augmentation techniques such as temporal scaling, spatial transformation, and noise injection, the system generates diverse training samples from limited annotated data, reducing the need for extensive manual data collection while improving model generalization.
2Measurement precision
If more supervised data is collected for training, then model estimation accuracy is improved, but data collection cost and complexity increase
Solution Approach 1:
The system enables self-service learning by automatically generating movement templates and annotated data from raw video sequences without requiring manual intervention. The unsupervised movement extraction algorithm autonomously identifies action boundaries and generates training labels, allowing the system to improve its performance through self-annotation rather than relying on expensive human annotators.
Solution Approach 2:
The system changes the parameters of training data by applying various transformations to movement templates, including temporal scaling, speed variation, and spatial transformations. This approach creates diverse training samples from a single annotated example, effectively increasing the training dataset size without additional data collection efforts.
3Manufacturing precision
If manual annotation of action segments is performed, then training data quality is improved, but processing time and labor cost increase
Solution Approach 1:
The system segments the data processing task into two distinct phases: unsupervised movement template extraction and supervised model training. By separating the template extraction phase (which can be automated) from the model training phase (which benefits from quality annotated data), the system achieves both high automation and high data quality. The segmentation allows different processing strategies to be applied to different parts of the pipeline.
Solution Approach 2:
The system introduces movement templates as an intermediary representation between raw video data and the final action segment estimation model. These templates serve as a bridge that captures essential movement patterns without requiring full manual annotation of action segments. The intermediary templates can be generated automatically while still providing sufficient structure for accurate supervised learning.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In a hidden semi-Markov model, observation probabilities for each type of movement of plural first hidden Markov models are learned using unsupervised learning. The hidden semi-Markov model includes plural second hidden Markov models each containing plural of the first hidden Markov models using types of movement of a person as states and with the plural second hidden Markov models each using actions determined by combining plural of the movements as states. The learnt observation probabilities are fixed, input first supervised data is augmented so as to give second supervised data, and transition probabilities of the movements of the first hidden Markov models are learned by supervised learning in which the second supervised data is employed. The learnt observation probabilities and the learnt transition probabilities are used to build the hidden semi-Markov model that is a model for estimating segments of the actions. Augmentation is performed on the first supervised data by adding teacher information of the first supervised data to each item of data generated by at least one out of oversampling in the time direction or oversampling in feature space.