Pedestrian Activity Recognition With Meta-Learning and Limited Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for recognizing and interpreting pedestrian activities around vehicles require large annotated datasets and rely on time-consuming, expensive human-labeled data, which limits their ability to generalize to new conditions, especially for autonomous or semi-autonomous vehicles.
Innovation Solution
A Siamese neural network is trained using recordings from multiple image capture devices to recognize pedestrian activities in an unsupervised manner, creating activity clusters and automatically annotating data, allowing for efficient pedestrian activity recognition and prediction with limited data, enabling automatic vehicle maneuvers based on predicted activities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large annotated datasets are used to train supervised machine learning models, then pedestrian activity recognition accuracy is improved, but data annotation time and cost increase
Solution Approach 1:
The system uses semi-supervised learning where the model automatically annotates its own training data by leveraging confidently classified samples from unlabeled data. This self-annotation process eliminates the need for extensive manual human labeling while maintaining recognition accuracy through iterative training on automatically generated annotations.
Solution Approach 2:
The system performs preliminary clustering of unlabeled pedestrian data using unsupervised learning algorithms before formal training. This preliminary organization of data into meaningful groups enables the subsequent supervised training to be more efficient and accurate, reducing the amount of manual annotation needed while improving recognition performance.
2Measurement precision
If hand-tuned human-motion based models are used, then pedestrian activity classification is improved, but generalization to new conditions deteriorates
Solution Approach 1:
The system employs a dynamic training process where the model is iteratively retrained on newly annotated data from diverse conditions and environments. This continuous adaptation allows the model to generalize better to new situations while maintaining high classification accuracy, overcoming the rigidity of static hand-tuned models.
Solution Approach 2:
The semi-supervised learning framework creates a universal model that can handle multiple pedestrian activity types and varying conditions through a unified training approach. The model learns from diverse data sources and automatically adapts to different scenarios, making it versatile across various driving conditions and pedestrian behaviors.
3Manufacturing precision
If extensive human labeling is performed, then training data quality is improved, but processing cost increases
Solution Approach 1:
The system automatically generates high-quality training annotations through semi-supervised learning, where the model identifies confidently classified samples from unlabeled data and uses them for self-training. This automated annotation process maintains training data quality while eliminating the need for expensive manual human labeling.
Solution Approach 2:
The system introduces an automatic annotation mechanism as an intermediary between raw unlabeled data and the training process. This intermediary layer uses confidence-based selection and automatic labeling to produce high-quality training data without direct human intervention, significantly reducing processing costs while maintaining data quality.
Data Source
AI summary
Pedestrian activity recognition is embodied in a method, system, non-transitory computer-readable and vehicle. A Siamese neural network is trained to recognize a plurality of pedestrian activities by training it recordings of the same pedestrian activity from two or more separate training image capture devices. The Siamese neural network is deployed with continual data collection from an additional image capture device to create a dataset of clusters of similar activities in an unsupervised manner. A spatio-temporal intent prediction model is then trained that can be deployed to recognize and predict pedestrian activity. Based on the likelihood of a particular pedestrian activity occurring or currently being underway, an automatic vehicle maneuver can be executed to navigate the situation.