Neural Sensor Fusion for Dense Occupancy Grid Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor systems in advanced driver assistance systems face challenges in generating dense environmental information efficiently, as low-cost sensors provide sparse data while high-cost sensors like LIDAR are not suitable for mass market applications, and smart sensors lack raw image data processing.
Innovation Solution
A neural network device and method for sensor fusion that combines feature-level data from multiple sensors to generate a dense environmental model, using unsupervised training to process sparse input data and classify large unobserved regions without requiring continuous information gathering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LIDAR sensors are used to provide dense environmental information, then measurement precision is improved, but device cost increases
Solution Approach 1:
The patent creates a virtual copy of dense LIDAR environmental data by training a neural network on LIDAR data during an offline phase. The trained network then generates synthetic dense environmental information from sparse radar and camera inputs, effectively copying the output characteristics of LIDAR without requiring physical LIDAR hardware in the deployment system.
Solution Approach 2:
The patent replaces expensive LIDAR sensors with cheaper radar and camera sensors that provide sparse data. By using cost-effective sensors and compensating through neural network processing, the system achieves dense environmental modeling capability without the high hardware cost of LIDAR.
2Measurement precision
If data is gathered over time to obtain meaningful environmental model, then measurement precision is improved, but productivity decreases
Solution Approach 1:
The patent performs preliminary training of the neural network offline using dense LIDAR data and sparse radar/camera data pairs. This pre-computed knowledge is stored in the network weights, enabling the system to generate dense environmental models instantly during runtime without requiring time-consuming data gathering or processing at that moment.
Solution Approach 2:
The patent replaces the mechanical approach of gathering data over time with multiple sensors with a computational approach using a trained neural network. The network directly transforms sparse sensor inputs into dense environmental models through learned mappings, eliminating the need for temporal data accumulation.
3Device complexity
If smart sensors are used to process detected data within the sensor, then device complexity is reduced, but loss of information increases
Solution Approach 1:
The patent introduces a neural network as an intermediary processing layer between the smart sensors and the environmental model generation. The network receives processed data from smart sensors and reconstructs dense environmental information that would otherwise be lost, acting as a mediator that recovers information rather than requiring access to原始 raw data.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A neural network device and a method using a neural network for sensor fusion are disclosed, wherein the neural network device includes a neural network (136) and wherein the neural network (136) is configured to: process a first grid (132) including a plurality of grid cells, wherein the first grid (132) represents at least a first portion of a field of view (108) of a first sensor (104), wherein at least one grid cell has information about an occupancy of the first portion of the field of view (108) assigned to the at least one grid cell, the information being based on data provided by the first sensor (104); process a second grid (134) including a plurality of grid cells, wherein the second grid (134) represents at least a second portion of a field of view (110) of a second sensor (106), wherein at least one grid cell has information about an occupancy of the second portion of the field of view (110) assigned to the at least one grid cell, the information being based on data provided by the second sensor (106); and fuse the processed first grid (132) with the processed second grid (134) into a fused grid (138), wherein the fused grid (138) includes information about the occupancy of the first portion of the field of view (108) of the first sensor (104) and the occupancy of the second portion of the field of view (110) of the second sensor (106).