Calibration-Linked Sensor Graph Fusion for Automotive Perception

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for processing sensor data from multiple automotive sensors result in information loss and increased computational effort due to the need for grid formation and limited inclusion of relative distances and locational information from all sensors.

Innovation Solution

A method for generating a fused automotive sensor data graph by combining multiple sensor data graphs using calibration matrices, which retains all sensor data nodes and edges, and introduces fusion edges to connect nodes across different graphs, enabling comprehensive data fusion and reduced training effort for machine learning models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If sensor data from multiple automotive sensors is combined in a two-dimensional grid or three-dimensional grid, then joint processing of sensor data using image processing approaches is enabled, but storage requirements increase and processing times are prolonged due to empty cells that need to be processed

Engineering Contradiction:

Improvejoint processing capabilityVSAvoidprocessing time

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent segments the sensor data processing by creating separate sensor data graphs for each automotive sensor instead of combining all sensors into a single grid. Each graph contains nodes representing sensor data points and edges representing spatial relationships, allowing independent processing of each sensor's data without the overhead of processing empty cells in a unified grid structure.

Inventive Principle:

Principle #1Segmentation

2Ease of operation

Engineering Contradiction:

Improvejoint processing capabilityVSAvoidstorage requirements

Core Design Contradiction:

Ease of operationVSQuantity of substance

Solution Approach 1:

The patent segments the sensor data processing by creating separate sensor data graphs for each automotive sensor instead of combining all sensors into a single grid. Each graph contains nodes representing sensor data points and edges representing spatial relationships, allowing independent processing of each sensor's data without the overhead of storing and processing empty cells in a unified grid structure.

Inventive Principle:

Principle #1Segmentation

3Ease of operation

If sensor data from multiple automotive sensors is combined in a two-dimensional grid or three-dimensional grid, then joint processing of sensor data using image processing approaches is enabled, but information loss occurs due to discretization of sensor data

Engineering Contradiction:

Improvejoint processing capabilityVSAvoidinformation loss due to discretization

Core Design Contradiction:

Ease of operationVSLoss of information

Solution Approach 1:

The patent segments the sensor data processing by creating separate sensor data graphs for each automotive sensor instead of combining all sensors into a single grid. Each graph contains nodes representing sensor data points and edges representing spatial relationships, allowing independent processing of each sensor's data without the overhead of storing and processing empty cells in a unified grid structure.

Inventive Principle:

Principle #1Segmentation

4Loss of information

If each node of the concatenated automotive sensor graph includes sensor data of various sensors and various sensor types, then comprehensive data fusion is achieved, but training the machine model requires multiple iterations

Engineering Contradiction:

Improvecomprehensive data fusionVSAvoidtraining time

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The patent segments the sensor data representation by creating separate sensor data graphs for each sensor type, where each graph maintains its own nodes and edges. This segmentation allows the machine learning model to process each sensor type's data structure independently, preserving the specific spatial relationships and data characteristics of each sensor type while enabling comprehensive multi-sensor fusion through the fusion graph structure.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentEP4502966A1Automotive perception based on a fused sensor graph

Publication Date: 2025.02.05 BAYERISCHE MOTOREN WERKE AG

Figure 1
Figure 2
Figure 3A~3B

AI summary

The present disclosure relates to a a method for generating a fused automotive sensor data graph as well as an automotive control unit configured to perform the method and a vehicle comprising the automotive control unit. A plurality of automotive sensor data graphs is obtained, which are each based on sensor data captured by a corresponding automotive sensor of the vehicle. Further, at least one calibration matrix is obtained, which defines a transformation between sensor data nodes of at least two automotive sensor data graphs. Based on the plurality of automotive sensor data graphs and the at least one calibration matrix, the fused automotive sensor data graph is generated, which comprises the plurality of automotive sensor data graphs as well as a plurality of fusion edges. The fused automotive sensor data graph is provided to an automotive perception function, which is implemented by a graph neural network.