Anomaly Detection in Image Recognition Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models in computer vision applications, such as image recognition algorithms, face challenges in distinguishing between real and false outliers during anomaly detection, which can lead to incorrect conclusions about the functionality of the model, especially in safety-critical systems like automotive systems where ISO standards 26262 and 21448 require stringent functional safety.
Innovation Solution
A method is introduced that involves a monitoring model to validate the functionality of a function model by performing a 'built-in' self-test (BIST) and pipeline validation, using reference inputs and hash values, to differentiate between correct and incorrect executions, and to ensure the integrity of communication between instances, thereby enhancing anomaly detection and overall system safety.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly detection is performed using a monitoring model to check for outliers in machine learning model outputs, then the ability to detect potential errors is improved, but the system cannot reliably distinguish between real errors and false outliers caused by rare input events
Solution Approach 1:
The patent segments the anomaly detection process into multiple independent components: (1) a monitoring model that detects outliers in model outputs, (2) a validation model that determines whether detected outliers represent real errors or false positives, and (3) a differentiation mechanism that distinguishes between model failures and rare but valid input cases. This segmentation allows each component to specialize in a specific aspect of error detection, improving overall reliability without sacrificing precision.
Solution Approach 2:
The patent introduces an intermediary validation mechanism between the monitoring model and the final error conclusion. The validation model acts as a mediator that receives outlier detections from the monitoring model and performs additional analysis to determine whether these outliers represent genuine model errors or are merely responses to rare but valid input scenarios. This intermediary layer resolves the contradiction by adding a decision-making step that preserves both detection sensitivity and precision.
2Reliability
If a critical path is explicitly defined and secured for safety-critical systems, then functional safety requirements are met, but the system complexity increases and not all components can be made fully safe according to ISO 26262
Solution Approach 1:
The patent segments the safety assurance mechanism into a dedicated validation model that operates independently from the main machine learning model. This validation model is specifically designed to handle safety-critical verification tasks, allowing the rest of the system to operate with standard components. The segmentation enables focused safety implementation without requiring complete system redesign, thus managing complexity while maintaining functional safety.
Solution Approach 2:
The validation model serves as an intermediary safety layer between the machine learning model and the control system. It mediates the interaction by verifying model outputs before they reach critical control functions, thereby protecting the system without requiring all components to be fully safety-certified. This intermediary approach meets functional safety requirements while allowing integration of non-certified components in the broader system.
3Productivity
If machine learning models are used in safety-critical applications, then the functionality and performance are improved, but the risk of unsafe operations increases due to potential model or hardware failures
Solution Approach 1:
The patent implements preliminary safety actions by introducing a validation model that performs pre-verification of machine learning model outputs before they are used in safety-critical control decisions. This preliminary check catches potential errors early in the processing chain, preventing unsafe operations from occurring. The validation model proactively identifies and flags suspicious outputs, allowing the system to take preventive measures before actual safety incidents can occur.
Solution Approach 2:
The patent establishes a feedback mechanism where the validation model continuously monitors and evaluates the outputs of the machine learning model, providing feedback about detected anomalies and potential errors. This feedback loop enables real-time safety assessment and allows the system to adjust its operations based on the validation results. The feedback mechanism ensures that safety concerns are addressed promptly, maintaining both high functionality and reliability in safety-critical applications.
Data Source
AI summary
System and computer-implemented method for monitoring a function model for providing data for at least one function of a computer-controlled machine, in particular an image recognition algorithm.