Temporal Object Intention Classification With Cached ML Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer vision systems are incapable of detecting enough information to operate complex systems such as autonomous vehicles, and they suffer from high computational load.
Innovation Solution
A machine-learned (ML) architecture that processes temporal sensor data by associating multiple images of the same object over time, using a cache to store intermediate outputs and reducing the computational load through a series of ML layers to determine confidence scores for object attributes and intentions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional computer vision techniques are used for object detection and classification, then the system can identify salient portions of images, but the computational load becomes excessively high and detection capabilities are insufficient for complex systems
Solution Approach 1:
The patent segments the computer vision system into multiple specialized machine-learned components: an object detector for identifying salient portions, an attribute classifier for detecting object attributes, and an intention classifier for predicting object intentions. Each component processes specific aspects of the input data independently, improving overall detection capabilities while distributing computational load across modular units rather than requiring a single complex system to handle all tasks simultaneously
Solution Approach 2:
The patent introduces temporal dimensionality by processing sequences of images over time rather than analyzing single static images. The intention classifier receives temporal information about object movements and behaviors across multiple time steps, enabling prediction of future intentions. This temporal dimension enhances detection capabilities for dynamic scenes while the incremental processing of image sequences reduces the computational burden compared to analyzing all possible attributes simultaneously in each frame
2Measurement precision
If conventional computer vision techniques process images to identify salient portions, then basic object detection is achieved, but the system cannot detect object attributes or predict intentions with near-human-level perception
Solution Approach 1:
The patent divides the perception system into specialized components: an object detector for locating salient portions, an attribute classifier for detecting specific object attributes (such as pedestrian attributes like crossing intention, vehicle attributes like turning intention), and an intention classifier for predicting future behaviors. This segmentation allows each component to be optimized for its specific task, achieving near-human-level detection accuracy for particular attributes and intentions while keeping individual component complexity manageable
Solution Approach 2:
The patent introduces intermediate processing layers between basic object detection and final interpretation. The attribute classifier acts as an intermediary that extracts specific attributes from detected objects, and the intention classifier serves as another intermediary that predicts future behaviors based on temporal patterns. These intermediary components bridge the gap between simple detection and complex interpretation, achieving high measurement precision for specific attributes without requiring the entire system to be overly complex
3Measurement precision
If the system processes multiple images over time to improve detection accuracy, then detection capabilities increase, but computational bandwidth requirements increase
Solution Approach 1:
The patent extracts and processes only the most relevant information from multiple images. The object detector identifies salient portions in each image, and subsequent attribute and intention classifiers focus specifically on these detected objects rather than processing all pixels and features in every image. This extraction approach maintains high detection effectiveness by concentrating computational resources on relevant objects and attributes while reducing overall computational bandwidth requirements compared to exhaustive processing of all image data
Solution Approach 2:
The patent performs preliminary object detection and attribute classification before full intention prediction. The object detector first identifies salient portions in images, then the attribute classifier detects object attributes from these detected objects. Only after these preliminary actions are completed does the intention classifier predict future intentions based on temporal patterns. This staged approach allows the system to process multiple images over time effectively while managing computational bandwidth by performing less intensive operations first and reserving more intensive intention prediction for when necessary
Data Source
AI summary
A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.