Score Function Optimization for Imbalanced Two-Class Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In two-class classification, when the number of positive example class data is significantly smaller than the negative example class data, existing methods face challenges in preventing overlearning and achieving high classification accuracy, as they either lead to excessive adaptation of the score function to training data or limit the use of nonlinear models due to the requirement of two probability models.
Innovation Solution
A learning apparatus and method that optimize the score function by using a weighted combination of AUC and pAUC as an index, updating the score function parameters through machine learning while decreasing the weight λ, allowing for the use of nonlinear models like deep networks without overlearning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a learning method using only pAUC as an index is employed to handle imbalanced data, then classification accuracy on positive examples is improved, but overlearning occurs and versatility is lost
Solution Approach 1:
The patent combines AUC and pAUC into a composite evaluation index by linearly combining their gradient components. This merging allows the learning method to simultaneously consider both overall classification performance (AUC) and performance at specific false positive rates (pAUC), preventing overfitting to training data while maintaining high classification accuracy on positive examples.
2Reliability
If two probability models are required as in Patent Document 1, then overlearning is suppressed, but nonlinear models such as deep networks cannot be used
Solution Approach 1:
The patent changes the evaluation index from using two separate probability models to using a single probability model with a composite index that linearly combines gradients from both AUC and pAUC. This parameter change allows the use of flexible nonlinear models like deep networks while still suppressing overlearning through the dual-objective gradient combination.
Data Source
AI summary
The learning apparatus for machine learning a score function to be used for two-class classification includes a score function optimization unit that updates, by setting, as an index, a total value of a value obtained by multiplying an AUC by a weight λ which is a positive real number and a value obtained by multiplying a pAUC by a weight (1-λ), a parameter of the score function through machine learning that is performed, using training data, so as to maximize the index, while decreasing the value of the weight λ. The AUC indicates an area of a region on a horizontal axis side of a curve obtained by changing a threshold for determining a positive example and a negative example in the score function. The pAUC is a value of the AUC when the value of the false positive rate is set to a specific fixed value.