Distribution Shift Detection via Dual Processing Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for detecting distribution shifts in data and feature distributions, particularly in machine learning-based systems for motor vehicles, are inadequate due to the difficulty in evaluating functional quality and detecting slight inaccuracies or failures, especially in environments where complete failure is undesirable.
Innovation Solution
A method utilizing two structurally different processing modules, one created through machine learning and the other not, to compare results and detect distribution shifts by identifying deviations in object classification probabilities, with candidate signals generated and transmitted to a central device for further evaluation and adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single machine learning model is used for object classification, then the system is simple and energy-efficient, but it cannot detect distribution shifts or concept drift in changing environments
Solution Approach 1:
The patent applies parameter changes by comparing outputs from processing modules with different structural parameters (one trained via machine learning, another with fixed parameters). This allows detection of distribution shifts through parameter deviation without fundamentally changing the core classification function, resolving the contradiction between reliability improvement and complexity increase.
2Reliability
If machine learning models are continuously monitored for accuracy, then functional quality can be maintained, but complete failure detection is undesirable in safety-critical functions
Solution Approach 1:
The patent implements preliminary anti-action by detecting distribution shifts before they cause complete functional failure. The comparison between processing module outputs provides early warning signals that allow preventive measures to be taken, avoiding the harmful effect of complete functional failure while maintaining operational continuity.
3Measurement precision
If ground truth data is used to evaluate function accuracy during application, then distribution shifts can be detected, but ground truth is unavailable in real-world scenarios
Solution Approach 1:
The patent uses an intermediary approach by introducing a second processing module with fixed parameters as a reference system. This intermediary allows indirect measurement of distribution shifts through comparison, eliminating the need for direct ground truth data while maintaining detection precision.
4Measurement precision
If multiple processing modules are used to detect distribution shifts, then detection accuracy improves, but computing power and energy consumption increase
Solution Approach 1:
The patent applies partial action by using one machine learning processing module and one simpler processing module with fixed parameters for comparison. This partial configuration provides sufficient detection capability without the excessive energy consumption that would result from using multiple complex machine learning models, resolving the contradiction between measurement precision and energy usage.
Data Source
Figure 1~2
Figure 3
AI summary
The invention relates to a method for detecting a distribution shift in a data and/or feature distribution of input data (10), wherein the method is carried out in at least one mobile device (40), comprising the following steps: receiving the input data (10) by means of an input device (2), performing an identical function on the received input data (10) by means of a first processing module (3-1) and at least one second processing module (3-2), wherein the first processing module (3-1) and the at least one second processing module (3-2) are structurally different from each other, wherein at least the first processing module (3-1) was created on the basis of machine learning, comparing the results (11) supplied by the processing modules (3-1, 3-2) and determining a distribution shift based on the comparison result by means of an evaluation device (4).and if a distribution shift has been detected: providing a candidate signal (12), and outputting the provided candidate signal (12) by means of an output device (5). Furthermore, the invention relates to an associated device (1), a central unit (20) and a system (30).