Computer Vision Intent Inference Using Context Outside Bounding Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autonomous vehicles struggle to accurately infer the intent of entities, such as vehicles or pedestrians, based on sensor data, particularly under varying weather and lighting conditions, leading to potential misinterpretation of turn signals and brake lights.
Innovation Solution
A system is trained to infer intent using a model that integrates camera data with Lidar data, performing semantic and instance segmentation to accurately predict vehicle maneuvers by analyzing pixels outside bounding regions, incorporating contextual information and leveraging other machine learning models for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a model is trained to infer intent based on data inside bounding regions only, then the model training is simpler and faster, but the accuracy of inferring vehicle intents is insufficient
Solution Approach 1:
The patent segments the input data into two distinct parts: data inside bounding regions (primary object data) and data outside bounding regions (contextual data). This segmentation allows the model to process different types of information through separate pathways, improving intent inference accuracy by incorporating contextual information while maintaining manageable model complexity through structured processing.
2Reliability
If the system processes only data inside bounding regions, then the processing speed is faster and computational load is lower, but the system cannot accurately distinguish between turn signals and brake lights under varying environmental conditions
Solution Approach 1:
The system performs preliminary segmentation to identify and separate contextual data outside bounding regions before main processing. By pre-identifying relevant contextual information (such as environmental conditions, road geometry, and surrounding objects), the system prepares data in advance for more accurate intent inference, improving reliability in distinguishing turn signals from brake lights while optimizing computational resource usage.
3Measurement precision
If the model is trained using only camera data, then the system is simpler to implement, but the accuracy deteriorates under varying weather and lighting conditions
Solution Approach 1:
The patent merges camera data with data from other sensors (such as LiDAR, radar, or infrared sensors) to create a multi-modal input system. This combination allows the model to infer vehicle intent with higher accuracy under varying weather and lighting conditions by leveraging complementary information from different sensor types, while the modular architecture manages system complexity through integrated processing.
Data Source
AI summary
A system trains a model to infer an intent of an entity. The model includes one or more sensors to obtain frames of data, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform steps. A first step includes determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity. A second step includes identifying a common entity, the common entity being present in bounding regions corresponding to a plurality of the frames. A third step includes associating the common entity across the frames. A fourth step includes training a model to infer an intent of the common entity based on data outside of the bounding regions.