Orthogonal Data Shift for Machine Learning Model Deterioration Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to effectively identify the cause of accuracy deterioration in machine learning models due to domain shifts, making it difficult to implement countermeasures such as model re-learning.
Innovation Solution
A data presentation program and information processing device that acquires data from an estimation target dataset, shifts it in a direction orthogonal to the loss fluctuation in the feature space, and visualizes this shift to identify the cause of model deterioration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is presented in the direction of loss fluctuation, then the model's accuracy deterioration can be detected, but the cause of the domain shift cannot be effectively identified
Solution Approach 1:
The patent transforms the analysis from the original feature space to a new coordinate system where one axis represents the loss gradient direction and the other represents the orthogonal direction. By projecting data onto these transformed axes, the method separates the magnitude of loss change from the direction of feature variation, enabling identification of domain shift causes in the orthogonal direction while maintaining detection capability along the loss gradient.
Solution Approach 2:
The patent introduces gradient computation as an intermediary mechanism that bridges the relationship between input data and model loss. By calculating gradients with respect to input data and using them to define orthogonal directions, the method creates a mediator that reveals how specific feature variations contribute to loss changes, thereby identifying the causes of domain shift.
2Ease of operation
If conventional data presentation methods are used, then simple visualization is achieved, but the domain shift characteristics remain unclear
Solution Approach 1:
The patent applies dimensionality transformation by projecting data onto gradient-aligned coordinate systems. This transformation reorganizes the feature space such that the orthogonal complement of the gradient direction becomes explicitly visible, allowing domain shift characteristics to be observed in a structured manner while maintaining visualization clarity.
Solution Approach 2:
The patent employs visual encoding by representing different gradient directions and orthogonal directions through distinct visual indicators. This allows the complex relationships between feature variations and loss changes to be perceived intuitively, maintaining ease of operation while revealing domain shift characteristics through visual differentiation.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
A non-transitory computer-readable storage medium storing a data presentation program that causes at least one computer to execute a process, the process includes acquiring certain data from an estimation target data set that uses an estimation model, based on an estimation result for the estimation target data set; and presenting data obtained by changing the certain data in a direction orthogonal to a direction in which loss of the estimation model fluctuates, in a feature space that relates to feature amounts obtained from the estimation target data set.