ML Tool Recalibration Using Critical Instance Label Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) tools face challenges in adapting to changing environments due to dynamic production conditions, leading to inefficient and laborious recalibration processes that can introduce spurious correlations, catastrophic forgetting, and high computational costs, especially in edge devices.
Innovation Solution
A method for autonomously recalibrating ML-based tools by identifying critical data instances and generating a calibration signal to adapt the model using Explainable AI techniques, minimizing manual effort and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the ML model is retrained with new data in the changed environment, then the model adapts to the new environment, but the adaptation becomes laborious and may introduce spurious correlations or catastrophic forgetting
Solution Approach 1:
The patent segments the recalibration process by identifying and focusing only on critical data instances that cause mismatches between transferred labels and model predictions, rather than retraining with all new data. This segmentation reduces the recalibration workload while maintaining adaptation effectiveness.
Solution Approach 2:
The patent extracts and removes harmful elements from the recalibration process by eliminating the need for extensive new data collection and labeling. Instead, it uses label transfer from source domain and selectively processes only critical instances, taking out the laborious and error-prone aspects of traditional retraining.
2Adaptability or versatility
If extensive new data is collected and used for retraining, then the model adapts to the new domain, but the computational costs and data labeling effort increase significantly
Solution Approach 1:
The patent performs preliminary label transfer from the source domain to the target domain before recalibration. By pre-transferring labels and identifying critical instances in advance, the method eliminates the need for time-consuming data collection and manual labeling during the adaptation phase.
Solution Approach 2:
The patent uses label copying from the source domain to the target domain through label transfer. Instead of creating new labels through expensive manual annotation, it copies and adapts existing labels, significantly reducing the time and resources required for data preparation.
3Adaptability or versatility
If the ML model is adapted without constraints from the previous task, then the model learns new behavior, but catastrophic forgetting occurs and the model no longer works well on the previous task
Solution Approach 1:
The patent applies local quality by making targeted adjustments only to critical components and critical data instances rather than globally retraining the entire model. This localized recalibration allows the model to learn new behavior in specific areas while preserving its performance on previous tasks.
Solution Approach 2:
The patent uses partial action by performing recalibration on only a subset of critical data instances rather than the entire dataset. This partial recalibration is sufficient to adapt to the new domain while minimizing disruption to previously learned behaviors and preventing catastrophic forgetting.
4Adaptability or versatility
If naive data gathering and labelling is used for model adaptation, then the model can be retrained, but the process opens the door for learning brittle behavior and requires extensive testing
Solution Approach 1:
The patent introduces label transfer as an intermediary mechanism between the source domain and target domain. This intermediary process provides structured and reliable label propagation, avoiding the brittleness of naive data gathering and manual labeling while maintaining model adaptability.
Data Source
Figure 1~2
Figure 3
AI summary
Assistance apparatus and computer-implemented method for autonomously recalibrating an ML-based tool (Ms) comprising a machine learning model which is trained with sensor data collected from a source system, comprising the steps: - receiving (S1) a) source data instances (SD) collected from the source system, b) information (INF) about the at least one changed characteristic in the source system, c) target data instances (TD) collected in a target system, the target system is configured with at least one changed characteristic compared to the source systems, - transferring (S2) labels (L) from the source data in-stances (SD) to the target data instances (TD) using a label transfer tool, - identifying (S3) critical target data instances (cTD) where predictions generated by the ML-based tool for the target data instances (TD) mismatch with the transferred labels for the target data instances (TD), - identifying (S4) at least one critical component (cC) of the critical target data (cTD) instances that leads to the mismatch, - generating (S5) a calibration signal (CS) depending on the critical component (cC) and the information (INF) about the changed characteristics in the target system, - recalibrating (S6) the ML-based tool (Ms) using the calibration signal (CS) resulting in a target ML-based tool, and applying (S7) the target ML-based tool in the target system (Mt).