Decision Tree Training Data Influence Evaluation With Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for evaluating the influence of training data on prediction accuracy in decision tree type machine learning models are inefficient, as they either require significant processing time or are limited to specific types of models, such as deep learning models.
Innovation Solution
A computer system that calculates a similarity score for each training data point based on a decision tree structure and evaluates an influence score to identify training data that affects prediction accuracy, while minimizing processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If reference models are learned for all training data to evaluate influence degree, then evaluation accuracy is improved, but processing time becomes great
Solution Approach 1:
The patent segments the training data evaluation process into two distinct phases: (1) a similarity score calculation phase that computes similarity between each training data point and others using the trained model's tree structure, and (2) an influence score calculation phase that only evaluates target data selected based on similarity thresholds. This segmentation avoids the need to learn reference models for all training data while maintaining evaluation accuracy for the most influential data points.
Solution Approach 2:
The patent applies partial action by selecting only a subset of training data (target data) for influence score evaluation based on similarity scores, rather than evaluating all training data. By setting a similarity threshold, the system focuses computational resources on evaluating only those data points that are most similar to the query data, thereby reducing processing time while maintaining evaluation accuracy for critical cases.
2Productivity
If deep learning model characteristics are used to evaluate influence, then evaluation speed is improved, but applicability is limited to deep learning models only
Solution Approach 1:
The patent creates a universal evaluation method that works across different machine learning model types by leveraging the common concept of similarity in feature space. The similarity score calculation using the trained model's structure (tree structure for decision trees, or other model structures) provides a model-agnostic approach that can be applied to decision trees, deep learning models, and other types, thereby achieving both speed and versatility.
Solution Approach 2:
The patent introduces similarity score as an intermediary metric that bridges the gap between different model types. Instead of directly using model-specific characteristics (which would limit applicability), the similarity score serves as a universal mediator that captures the essential relationship between training data points in a way that is applicable across different model architectures while still enabling efficient evaluation.
3Loss of time
If similarity score calculation using tree structure is performed, then processing time is reduced, but evaluation precision may be affected
Solution Approach 1:
The patent performs preliminary similarity score calculation using the trained model's tree structure before conducting the actual influence score evaluation. This preliminary action filters out data points that are dissimilar to the query data, allowing the system to skip expensive reference model learning for those points while maintaining precision for the selected target data where full evaluation is performed.
Data Source
AI summary
A computer system is provided, which is capable of evaluating the degree of influence of training data on prediction accuracy of a decision tree type machine learning model, while suppressing increase in processing time thereof. A similarity score calculating unit uses a tree structure of a trained model of a target predictor to calculate, for each of training data used for learning this trained model, a similarity score in which is evaluated similarity between the training data in the trained model and other training data. An evaluating unit selects target data that is training data that is a target of evaluation from the training dataset on the basis of the similarity score, and calculates an influence score in which the degree of influence of the target data on accuracy of the trained model is evaluated.


