Gain Matrix Loss Function for Imbalanced Data Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In machine learning, particularly for abnormality detection and medical image diagnosis, training data sets often exhibit class imbalance, making it difficult to evaluate classifier performance using traditional metrics, and existing methods underutilize reliable unlabeled training data in cost-sensitive learning and semi-supervised learning scenarios.
Innovation Solution
An information processing apparatus that selects and utilizes unlabeled training data with reliable predicted labels by applying a Kullback-Leibler divergence condition and incorporates a gain matrix-based loss function to train machine learning models, ensuring effective use of both labeled and unlabeled data for optimization, even in challenging metric scenarios like worst recall and precision constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional accuracy metric is used for evaluation, then simplicity of evaluation is maintained, but performance evaluation becomes biased toward majority class in imbalanced data sets
Solution Approach 1:
The patent transforms the evaluation metric from simple accuracy to a gain-based metric that incorporates class-specific costs and benefits. By changing the parameter from accuracy to gain (which considers true positive gain, false positive cost, false negative cost, and true negative gain), the system achieves both simplified optimization through cross-entropy loss and accurate performance evaluation for imbalanced datasets.
2Measurement precision
If complicated metrics like worst recall or precision constraints are used, then performance evaluation becomes more accurate for imbalanced data, but optimization becomes difficult and computationally expensive
Solution Approach 1:
The patent introduces a gain matrix as an intermediary that bridges the gap between complex performance metrics and simple optimization. The gain matrix encodes class-specific costs and benefits, allowing the system to optimize using simple cross-entropy loss while indirectly controlling complex metrics like recall and precision through the gain parameters.
Solution Approach 2:
The patent changes the optimization parameter from complex constraints (worst recall, precision thresholds) to a gain matrix with class-specific weights. This parameter transformation allows the same optimization framework to achieve both simple computation and accurate control over performance metrics for imbalanced datasets.
3Ease of manufacture
If only labeled training data is used, then training process is simple, but reliable unlabeled data is underutilized leading to suboptimal model performance
Solution Approach 1:
The patent enables the model to self-improve by utilizing its own predictions on unlabeled data. Through pseudo-labeling, the model generates predicted labels for unlabeled data, and these pseudo-labels are used to create additional training samples. This self-service mechanism allows the system to automatically leverage unlabeled data without complex manual annotation processes.
Solution Approach 2:
The patent performs preliminary action by generating pseudo-labels for unlabeled data before final training. The system first creates predicted labels for unlabeled samples, then uses these pseudo-labels to construct augmented training datasets, which are subsequently used in the training process to improve model performance.
Data Source
AI summary
An information processing apparatus includes one or more memories; and one or more processors coupled to the one or more memories, the one or more processors being configured to decide a gain matrix based on an input metric, perform selection of first training data from a plurality of unlabeled training data, to be used for training a machine learning model, based on the gain matrix, and perform training of the machine learning model based on the first training data, a predicted label that is predicted from the first training data, and a loss function including the gain matrix.