Pedestrian Gesture Detection Using Multi-Sensor Temporal Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision technologies for autonomous vehicles struggle to accurately detect pedestrian attributes and gestures, particularly in high-risk environments, which can lead to uninformed and potentially unsafe decisions.
Innovation Solution
The use of machine-learned models to detect pedestrian attributes and gestures from sensor data, such as image data, lidar data, and radar data, allowing for the determination of classifications, activities, poses, and gestures of pedestrians, and subsequently controlling the vehicle's operation based on this information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine-learned models are used to detect pedestrian attributes and gestures, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The machine-learned model is divided into a backbone network for feature extraction and a temporal head for gesture classification. This segmentation allows each component to specialize in specific tasks, improving overall detection accuracy while managing computational complexity through modular architecture.
Solution Approach 2:
The system transitions from analyzing single static images to processing sequences of images across the temporal dimension. By incorporating temporal information from multiple time steps, the model achieves more accurate gesture detection while the modular architecture helps manage the increased computational requirements.
2Reliability
If multiple sensor data types are processed, then reliability is improved, but loss of time increases
Solution Approach 1:
The system merges multiple sensor data types (image data, LiDAR data, radar data) into a unified processing pipeline. By combining these data sources and processing them through the same machine-learned model architecture, the system improves detection reliability through multi-sensor fusion while avoiding redundant processing steps that would increase time loss.
Data Source
AI summary
Techniques for detecting attributes and/or gestures associated with pedestrians in an environment are described herein. The techniques may include receiving sensor data associated with a pedestrian in an environment of a vehicle and inputting the sensor data into a machine-learned model that is configured to determine a gesture and/or an attribute of the pedestrian. Based on the input data, an output may be received from the machine-learned model that indicates the gesture and/or the attribute of the pedestrian and the vehicle may be controlled based at least in part on the gesture and/or the attribute of the pedestrian. The techniques may also include training the machine-learned model to detect the attribute and/or the gesture of the pedestrian.