Overfitting Detection in Machine Learning Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques struggle to objectively identify the factors causing overfitting in machine learning models, even when the contribution degree of features is visualized, making it difficult to understand which input values contribute significantly to prediction.
Innovation Solution
An information processing apparatus that outputs a first object region in an input image using a machine learning model, specifies a basis region where image features have a higher contribution degree than a threshold, and executes an object detection process in a mask image to determine if the first object is detected again, thereby suggesting potential factors causing overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the machine learning model is learned by using a small amount of training data or biased training data, then the learning process is faster and requires fewer resources, but the model enters an over-fitted state causing erroneous detection or detection omission
Solution Approach 1:
The patent applies preliminary action by performing overfitting detection before the model is deployed for actual use. The detection process uses verification data to identify overfitting states in advance, allowing the system to prevent erroneous detections and detection omissions before they occur in production, thus maintaining reliability without sacrificing learning speed
Solution Approach 2:
The patent implements feedback by using verification data to evaluate the model's performance and detect overfitting states. The system continuously monitors detection results and uses this feedback to identify when the model has overfitted, enabling corrective actions to be taken to maintain detection accuracy
2Loss of information
If the contribution degree of features is visualized using existing techniques, then some information about feature importance is provided, but it is difficult to objectively grasp what kind of factors cause the overfitting
Solution Approach 1:
The patent applies segmentation by dividing the verification data into multiple groups with different characteristics. By analyzing each group separately and comparing results, the system can identify specific factors causing overfitting, transforming the abstract problem of overfitting into manageable, analyzable segments that reveal causal relationships
Solution Approach 2:
The patent uses local quality by analyzing feature contribution degrees in different regions or contexts of the verification data. Instead of treating all data uniformly, the system examines local patterns and characteristics to identify specific factors that cause overfitting in particular scenarios, providing granular insights into overfitting causes
Data Source
AI summary
An information processing apparatus detects overfitting of a machine learning model. The information processing apparatus outputs a first object region indicating a region of a first object in an input image by using the machine learning model for detecting an object in an image, and specifies, as a basis region, a region in which a contribution degree of each of image features is higher than a predetermined threshold value when the machine learning model outputs the first object region. The information processing apparatus executes an object detection process in a mask image, and determines whether or not the first object is detected again by using the machine learning model, the mask image obtained by invalidating the image features distributed in a region different from the first object region in the basis region in the input image.