Multimodal Sensor Fusion With Self-Healing Fault Compensation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Autonomous vehicles rely on a minimal set of sensors for safe operation, but current systems lack robustness as they depend on each sensor type, leading to emergency stops when one sensor malfunctions, and addressing this requires redundant sensors, increasing cost and complexity.

Innovation Solution

A multimodal sensing approach with self-healing capabilities that utilizes multiple sensors to detect and compensate for sensor deficiencies, allowing for continuous operation by maximizing information fusion and maintaining classification accuracy through parallel classification processes and sensor monitoring, thereby reducing the need for redundant sensors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If redundant sensors are provided to ensure safety and reliability, then the reliability of autonomous vehicle operation is improved, but the device complexity and manufacturing cost increase

Engineering Contradiction:

Improveautonomous vehicle operation safetyVSAvoidsensor system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system performs self-diagnosis by continuously monitoring sensor confidence levels and automatically detecting malfunctions without external intervention. The sensor suite supervisor process enables the system to self-heal by identifying and compensating for sensor failures in real-time, eliminating the need for manual inspection and maintenance while maintaining high reliability

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements continuous feedback loops where sensor data is constantly monitored, confidence levels are evaluated, and malfunction detection triggers automatic compensation mechanisms. This closed-loop feedback system ensures reliable operation by dynamically adjusting to sensor failures and maintaining accurate environmental perception

Inventive Principle:

Principle #23Feedback

2Reliability

If redundant sensors are provided to ensure safety and reliability, then the reliability of autonomous vehicle operation is improved, but the manufacturing cost increases

Engineering Contradiction:

Improveautonomous vehicle operation safetyVSAvoidmanufacturing cost

Core Design Contradiction:

ReliabilityVSEase of manufacture

Solution Approach 1:

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

Instead of providing full redundant sensor sets, the system applies partial redundancy by using machine learning models to compensate for single sensor failures. This approach achieves sufficient reliability for safe operation while significantly reducing the number of physical sensors required, thereby lowering manufacturing costs

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If sensor monitoring and self-healing processes are implemented, then the reliability of operation is improved, but the processing time and computational load increase

Engineering Contradiction:

Improvecontinuous operation capabilityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-trains machine learning models during the development phase to handle various sensor failure scenarios. This preliminary preparation enables the runtime system to quickly compensate for sensor malfunctions using pre-computed compensation strategies, minimizing processing delays during actual operation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The monitoring and compensation process is segmented into independent parallel operations: confidence level evaluation, malfunction detection, and compensation execution. This segmentation allows each component to operate independently and efficiently, reducing overall processing time while maintaining continuous monitoring capability

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11320810B2Multimodal sensing in autonomous driving vehicles with self-healing capabilities

Publication Date: 2022.05.03 INTEL CORP

AI summary

An apparatus for autonomous vehicles includes a perception pipeline having independent classification processes operating in parallel to respectively identify objects based on sensor data flows from multiple ones of a plurality of sensors. The apparatus also includes a sensor monitoring stage to operate in parallel with the perception pipeline and to use the sensor data flows to estimate and track a confidence level of each of the plurality of different sensors, and nullify a deficient sensor when the confidence level associated with the deficient sensor fails to meet a confidence threshold.