Mobile Platform Environment Representation for Transparent Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for representing the environment of mobile platforms, such as autonomous vehicles, lack a modular and transparent approach for fusing sensor data from multiple sources, leading to inefficiencies in perception and decision-making processes.
Innovation Solution
A modular method using a deep neural network to fuse discrete time sequences of sensor data from multiple sensors, transforming them into a unified moving spatial reference system, and generating a predictive environment representation using an encoder-decoder CNN.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional sensor fusion methods are used, then the system structure is simpler, but the transparency and interpretability of environment perception deteriorates
Solution Approach 1:
The patent segments the environment representation into discrete spatial cells (grid cells) that can be individually processed and interpreted. Each cell maintains independent semantic information, allowing the system to preserve interpretability while handling complex multi-sensor data through modular cell-based processing rather than monolithic fusion.
2Measurement precision
If more sensors are added to improve perception accuracy, then the measurement precision improves, but the device complexity increases
Solution Approach 1:
The patent creates a universal grid cell structure that can accommodate multiple sensor types (LIDAR, camera, radar) through a common representation framework. Each sensor type contributes to the same cell-based environment model, allowing diverse sensors to be integrated without proportionally increasing system complexity through standardized multi-functional processing.
3Measurement precision
If detailed environment representation is maintained for all objects, then the measurement precision improves, but the computational complexity increases with object density
Solution Approach 1:
The patent applies local quality by assigning different levels of detail and processing to different spatial cells based on their content and importance. High-priority cells (containing relevant objects) receive detailed processing while low-priority cells use simplified representations, maintaining overall accuracy while reducing computational complexity through localized quality adjustment.
Data Source
Figure 1
Figure 2
Figure 3a~3b
AI summary
A method and system for representing an environment of a first mobile platform is disclosed, the method comprising: capturing features of the environment (S1) by discrete time sequences of sensor-data originating from at least two sensors and respective time markers; determining distances (S2) of the first mobile platform to the features of the environment; estimating semantic information (S3) of the features of the environment; transforming the semantic information of the features of the environment into a moving spatial reference system (S4), wherein a position of the first mobile platform is at a constant site, using the respective determined distances and respective time markers; creating an input tensor (S5) using sequences of the transformed semantic information of the features of the environment, corresponding to the sequences of the sensor data of the at least two sensors; generating an output tensor (S6) that represents the environment using a deep neural network at a requested point in time and the input tensor, wherein the deep neural network is trained with training input tensors and true information training output tensors as to accurately reproduce arbitrary environment representations at the requested point in time.