Classification Model Learning With Constrained Recall Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing learning method for classification models struggles to achieve a high precision ratio while maintaining a high recall ratio, as emphasizing the recall ratio through increased weight β alters the tradeoff between precision and recall ratios, often resulting in a low precision ratio even with a high recall ratio.
Innovation Solution
An information processing system and method that learns a classification model by constraining the recall ratio to a minimum value, using a logistic regression model and optimizing the precision ratio through a constrained optimization problem, where the objective function minimizes false positives while ensuring a predetermined recall ratio is met.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a smaller threshold value is assigned to the classification probability, then the recall ratio is increased, but the precision ratio decreases due to increased false positives
Solution Approach 1:
The patent changes the optimization parameter from maximizing a weighted sum (F-measure) to maximizing precision ratio subject to a minimum recall ratio constraint. This fundamental parameter change in the objective function allows the system to achieve high precision while maintaining high recall, resolving the tradeoff between the two metrics.
Solution Approach 2:
Instead of the conventional approach of maximizing recall with a weight parameter β, the patent inverts the approach by making precision the primary optimization target and treating recall as a constraint. This inversion allows the system to prioritize precision while ensuring recall requirements are met, thereby resolving the contradiction.
2Reliability
If the recall ratio is emphasized by increasing the weight β, then the recall ratio improves, but the precision ratio becomes small due to altered tradeoff between precision and recall
Solution Approach 1:
The patent inverts the conventional F-measure optimization approach by making precision the primary objective and recall the constraint. This inversion eliminates the need for weight parameter β and its associated tradeoff problems, allowing simultaneous achievement of high precision and high recall.
Solution Approach 2:
The patent introduces dynamic adjustment capability where the recall ratio constraint can be flexibly set based on application requirements. This dynamic approach allows the system to adapt to different monitoring needs while maintaining optimal precision, resolving the static tradeoff inherent in weighted optimization methods.
3Reliability
If TP is increased to improve the recall ratio, then the recall ratio improves, but FP is also increased causing the precision ratio to decrease
Solution Approach 1:
The patent changes the optimization parameter from a weighted sum approach to a constraint-based approach where precision is maximized subject to minimum recall requirements. This parameter change fundamentally alters how TP and FP are balanced, allowing TP to increase for high recall while controlling FP growth through the precision optimization objective.
Data Source
AI summary
A classification model with a high precision ratio at a high recall ratio is learned. A classification model learning system (100) includes a learning data storage unit (110) and a learning unit (130). The learning data storage unit (110) stores pieces of learning data each of which has been classified as a positive example or a negative example. The learning unit (130) learns, by using the pieces of learning data, a classification model in such a way that a precision ratio of classification by the classification model is made larger under a constraint of a minimum value of a recall ratio of classification by the classification model.