Pedestrian Road-Crossing Intention Detection Using Image-Text Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting pedestrian road crossing intentions are inadequate due to their reliance on complex feature relations that are challenging to learn and result in unsatisfactory accuracy.
Innovation Solution
A computer vision system utilizing a neural network that learns to relate image embeddings to text caption embeddings, classifying pedestrian road crossing intentions by matching image embeddings to text captions and images of road crossing or non-crossing scenarios, employing a Contrastive Language-Image Pre-training (CLIP) model to enhance classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current methods for detecting pedestrian road crossing intentions are used, then the system can detect pedestrian intentions, but the accuracy is unsatisfactory due to reliance on complex feature relations that are challenging to learn
Solution Approach 1:
The patent replaces traditional mechanical vision systems that rely on complex feature extraction and relation learning with a vision-language model that uses natural language processing. The VLM encodes images and text into embedding spaces and compares them using similarity metrics, substituting the complex mechanical feature relation learning with a more straightforward embedding comparison approach that achieves higher accuracy.
2Measurement precision
If a vision language model with iterative learning is used, then classification accuracy improves, but computational resources and processing time increase
Solution Approach 1:
The patent performs preliminary encoding of both image and text inputs into embedding spaces before进行比较. By pre-processing the inputs into standardized embedding representations, the system avoids repeated complex computations during the similarity comparison phase, thereby reducing overall processing time while maintaining high classification accuracy through iterative learning.
Data Source
AI summary
A system for classifying a road crossing intention of a pedestrian includes a processor including a pretrained image encoder generating an image embedding based upon an input image. The system further includes a remote server receiving the image embedding. The remote server device further references a plurality of pretrained image and text embeddings each corresponding to either a positive road crossing intention or a negative road crossing intention. The remote server device further determines a plurality of proximity values evaluating whether the input image is closer to the positive road crossing intention or the negative road crossing intention, evaluating the image embedding against each of the pretrained embeddings. The remote server device further classifies a road crossing intention of the pedestrian based upon the plurality of proximity values. The system further includes generates a road crossing intention output based upon the road crossing intention of the pedestrian.