Sensor Fusion Occupancy Grids for Trajectory Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Secondary and low-level vehicle computing systems face resource constraints, limiting their functionality due to insufficient computing resources and input data limitations, particularly in accurately predicting object locations and types using sensor modalities like lidar and radar, which often result in false positives and negatives.
Innovation Solution
Fusing sensor data from lidar and radar modalities to create feature maps that identify non-drivable, over-drivable, and under-drivable objects, using a machine-learned model to generate an occupancy grid that validates vehicle trajectories, thereby improving object detection and prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensor data from multiple modalities is processed using traditional methods, then object detection can be performed, but false positives and negatives increase due to resource constraints in low-level systems
Solution Approach 1:
The patent segments the object detection task into distinct processing stages: generating individual sensor feature maps from different modalities (lidar, radar, camera), fusing these feature maps through element-wise operations, and then processing the fused representation. This segmentation allows low-level systems to perform lightweight fusion operations while reserving complex processing for high-level systems, thereby improving detection reliability without overwhelming computational resources.
Solution Approach 2:
The patent introduces feature maps as intermediary representations that bridge different sensor modalities. Instead of directly processing raw sensor data which is computationally intensive, the system converts sensor data into compressed feature maps that capture essential information. These feature maps serve as intermediaries that can be efficiently fused and processed, reducing the computational burden on low-level systems while maintaining detection accuracy.
2Productivity
If low-level systems independently verify trajectories without high-level system intervention, then resource consumption is reduced, but detection accuracy may deteriorate due to limited computing resources
Solution Approach 1:
The patent implements partial action by having low-level systems perform only the essential trajectory validation using fused sensor feature maps, rather than attempting complete autonomous verification. This partial validation handles routine cases efficiently, while complex or ambiguous situations are escalated to high-level systems for comprehensive analysis. This approach maintains productivity by avoiding full high-level system intervention while preserving measurement precision when needed.
Solution Approach 2:
The patent changes the parameter representation from raw sensor data to fused feature maps with specific characteristics (element-wise combinations, standardized formats). This parameter transformation enables low-level systems to perform trajectory validation with reduced computational requirements while maintaining sufficient accuracy for safety-critical decisions. The fused feature maps contain distilled information that is optimized for efficient processing.
3Reliability
If fused sensor feature maps are generated and processed, then object detection accuracy improves, but computing resources are consumed
Solution Approach 1:
The patent merges sensor feature maps from different modalities through element-wise fusion operations, combining information from lidar, radar, and camera systems into a unified representation. This merging occurs at the feature map level rather than processing individual sensor readings, significantly reducing computational overhead. The fused feature maps enable improved trajectory validation accuracy by integrating complementary information while consuming fewer resources than processing each sensor modality separately.
Data Source
AI summary
Techniques for fusing sensor data generated by different sensor modalities to improve object detections and object predictions determined by low-level systems of a vehicle. The techniques may include determining feature maps based on sensor data generated by different sensor modalities associated with a vehicle. In some examples, the feature maps may include at least a first feature map indicative of a location of an object in an environment of the vehicle and a second feature map indicative of elevation information associated with the object. The techniques may also include inputting the first feature map and the second feature map into a machine-learned model associated with the low-level system of the vehicle. In some examples, an output may be received from the machine-learned model that includes an occupancy grid, and the occupancy grid may exclude representation(s) associated with over-drivable object(s) and/or under-drivable object(s) that may be disposed in the environment.