Decision Tree Training Data Influence Evaluation With Similarity Scoring

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for evaluating the influence of training data on prediction accuracy in decision tree type machine learning models are inefficient, as they either require significant processing time or are limited to specific types of models, such as deep learning models.

Innovation Solution

A computer system that calculates a similarity score for each training data point based on a decision tree structure and evaluates an influence score to identify training data that affects prediction accuracy, while minimizing processing time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If reference models are learned for all training data to evaluate influence degree, then evaluation accuracy is improved, but processing time becomes great

Engineering Contradiction:
Improveevaluation accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the training data evaluation process into two distinct phases: (1) a similarity score calculation phase that computes similarity between each training data point and others using the trained model's tree structure, and (2) an influence score calculation phase that only evaluates target data selected based on similarity thresholds. This segmentation avoids the need to learn reference models for all training data while maintaining evaluation accuracy for the most influential data points.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by selecting only a subset of training data (target data) for influence score evaluation based on similarity scores, rather than evaluating all training data. By setting a similarity threshold, the system focuses computational resources on evaluating only those data points that are most similar to the query data, thereby reducing processing time while maintaining evaluation accuracy for critical cases.

Inventive Principle:
Principle #16Partial or excessive action

2Productivity

If deep learning model characteristics are used to evaluate influence, then evaluation speed is improved, but applicability is limited to deep learning models only

Engineering Contradiction:
Improveevaluation speedVSAvoidmodel applicability
Core Design Contradiction:
ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal evaluation method that works across different machine learning model types by leveraging the common concept of similarity in feature space. The similarity score calculation using the trained model's structure (tree structure for decision trees, or other model structures) provides a model-agnostic approach that can be applied to decision trees, deep learning models, and other types, thereby achieving both speed and versatility.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces similarity score as an intermediary metric that bridges the gap between different model types. Instead of directly using model-specific characteristics (which would limit applicability), the similarity score serves as a universal mediator that captures the essential relationship between training data points in a way that is applicable across different model architectures while still enabling efficient evaluation.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Loss of time

If similarity score calculation using tree structure is performed, then processing time is reduced, but evaluation precision may be affected

Engineering Contradiction:
Improveprocessing timeVSAvoidevaluation precision
Core Design Contradiction:
Loss of timeVSMeasurement precision

Solution Approach 1:

The patent performs preliminary similarity score calculation using the trained model's tree structure before conducting the actual influence score evaluation. This preliminary action filters out data points that are dissimilar to the query data, allowing the system to skip expensive reference model learning for those points while maintaining precision for the selected target data where full evaluation is performed.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12561399B2Computer system and data analysis method
Publication Date: 2026.02.24 HITACHI LTD
  • US12561399B2 patent drawing
  • US12561399B2 patent drawing
  • US12561399B2 patent drawing

AI summary

A computer system is provided, which is capable of evaluating the degree of influence of training data on prediction accuracy of a decision tree type machine learning model, while suppressing increase in processing time thereof. A similarity score calculating unit uses a tree structure of a trained model of a target predictor to calculate, for each of training data used for learning this trained model, a similarity score in which is evaluated similarity between the training data in the trained model and other training data. An evaluating unit selects target data that is training data that is a target of evaluation from the training dataset on the basis of the similarity score, and calculates an influence score in which the degree of influence of the target data on accuracy of the trained model is evaluated.