Pedestrian Intent Prediction via Pose Keypoint Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for determining pedestrian intent in autonomous vehicles rely on crude methods that fail to account for human body language and awareness, lacking sophistication in understanding human pose from sequential images, leading to inaccurate intent predictions.
Innovation Solution
A system that translates sequential images of pedestrians into keypoints to determine their pose, using a processor to aggregate these keypoints into vectors, which are then mapped to intent using pose-to-intent mappings that consider cultural gestures and surroundings, with sensors like cameras and LIDAR providing video feeds to an intent determination service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If crude and rudimentary methods are used to determine pedestrian intent (analyzing direction, speed, and distance), then the system complexity is low, but the measurement precision of intent prediction is poor
Solution Approach 1:
The system segments the pedestrian analysis into multiple components: extracting visual features (pose, gestures, body orientation), extracting contextual features (surroundings, environment), and combining these into a comprehensive intent prediction. This segmentation allows the system to process complex information in manageable parts, improving prediction accuracy without overwhelming system complexity
Solution Approach 2:
The system transitions from analyzing simple motion parameters (direction, speed, distance) to analyzing multi-dimensional human pose data including body orientation, gestures, and spatial relationships. This dimensional expansion enables the system to capture subtle intent signals that crude methods miss, significantly improving intent prediction accuracy
2Loss of information
If the system analyzes only basic motion parameters, then the ease of operation is high, but the loss of information about human body language and awareness is significant
Solution Approach 1:
The system performs preliminary extraction of pose keypoints and gestures from video frames before intent prediction. By pre-processing the visual data to identify body orientation, hand gestures, and posture, the system preserves rich body language information that would otherwise be lost, while organizing this information in a structured format that facilitates subsequent analysis
3Adaptability or versatility
If a static model is used for intent determination, then the device complexity is low, but the adaptability to different human gestures and cultural contexts is poor
Solution Approach 1:
The system employs dynamic pose-to-intent mappings that can adapt to different gestures, cultural contexts, and situational factors. Rather than using a fixed static model, the system dynamically adjusts its interpretation based on the specific pose configuration, surrounding context, and learned patterns from training data, enabling flexible and culturally-aware gesture interpretation
Solution Approach 2:
The system changes the parameters used for intent determination based on contextual factors. Different weights and thresholds are applied to various pose features depending on the situation, allowing the same gesture to be interpreted differently in different contexts. This parameter adaptation enables the system to handle cultural variations and situational nuances
Data Source
AI summary
A system and a method are disclosed for determining intent of a human based on human pose. In some embodiments, a processor obtains a plurality of sequential images from a video feed, and determines respective keypoints corresponding a human in each respective image of the plurality of sequential images. The processor aggregates the respective keypoints for each respective image into a pose of the human and transmits a query to a database to find a template that matches the pose by comparing the pose to a plurality of templates poses that translate candidate poses to intent, each template corresponding to an associated intent. The processor receives a reply message from the database that either indicates an intent of the human based on a matching template, or an inability to locate the matching template, and, in response to the reply message indicating the intent of the human, outputs the intent.