Autonomous Vehicle Sensor Fusion for Fault-Tolerant Object Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion systems for identifying external objects in autonomous driving face challenges when errors occur in individual sensors, leading to inaccurate object identification.
Innovation Solution
A vehicle control apparatus utilizing a first sensor (camera) and a second sensor (LiDAR) with a processor that employs multiple neural network models to analyze images and point clouds, determining similarity values and generating control signals for autonomous driving based on the outputs of these sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor fusion is used to determine external objects using camera and LiDAR, then measurement precision is improved, but reliability deteriorates when an error occurs in one sensor
Solution Approach 1:
The patent divides the object identification process into separate processing paths for camera data and LiDAR data, each with its own neural network model. This segmentation allows independent error handling and fallback mechanisms, where if one sensor fails, the other can still provide identification results.
Solution Approach 2:
The system dynamically changes the processing parameters based on sensor status. When one sensor is detected to have an error, the system adjusts by relying more heavily on the other sensor's data and modifies the fusion strategy accordingly, maintaining reliability while preserving measurement precision.
2Measurement precision
If multiple neural network models are used to process camera and LiDAR data, then object identification accuracy is improved, but device complexity increases
Solution Approach 1:
The patent designs neural network models that can process multiple types of input data (camera images and LiDAR point clouds) through a unified architecture. This multi-functionality reduces the need for entirely separate processing systems, thereby managing complexity while maintaining high identification accuracy.
Solution Approach 2:
The system introduces an intermediary fusion module that coordinates between the camera processing path and LiDAR processing path. This mediator manages the complexity by standardizing the integration of results from multiple neural networks, making the overall system more manageable despite having multiple models.
3Reliability
If sensor data fusion is implemented to ensure accurate object identification, then reliability is improved, but processing time increases
Solution Approach 1:
The system performs preliminary processing of camera and LiDAR data separately through dedicated neural network models before fusion. This preliminary action prepares the data in advance, allowing faster fusion and decision-making when needed, thus reducing overall processing time while maintaining reliability.
Solution Approach 2:
The patent implements a hierarchical fusion strategy where critical identification tasks use full sensor fusion for high reliability, while less critical tasks can rely on partial fusion or single-sensor data. This selective approach reduces average processing time while maintaining reliability for important decisions.
Data Source
AI summary
An apparatus for controlling autonomous driving of a vehicle is introduced. The apparatus may comprise a first sensor configured to capture an image and a second sensor configured to acquire a cluster of points. The apparatus may further comprise a memory storing multiple neural network models and a processor configured to process data from the sensors. The processor obtains a first value, indicating a score for the type of a point associated with the second sensor, by inputting the image to a first neural network model. The processor also obtains a second value, indicating a score for the same type, by inputting the cluster of points to a second neural network model. Using these values, the processor determines a similarity value among points in the cluster. Based on this similarity value, the apparatus outputs a selected value, generates a signal, and subsequently controls the vehicle's autonomous driving.