Record Importance Determination via Hierarchical Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Evaluating the importance of individual records in large datasets during model building is challenging, as existing methods focus on feature importance rather than record-level impact, which can be affected by noise and patterns varying across records.
Innovation Solution
The method employs hierarchical clustering to group records and assess their importance by comparing model accuracy with and without each group, using a leave-one-out approach to determine the impact of each data group on predictive modeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If feature importance techniques are used to evaluate features independently, then feature-level model explanation is achieved, but record-level importance cannot be determined
Solution Approach 1:
The patent segments the dataset into multiple clustered data groups based on feature similarities. Instead of evaluating all records uniformly, the method divides the data into distinct groups (G1, G2, G3, etc.) where each group shares common characteristics. This segmentation enables record-level importance analysis by evaluating each group's contribution to model accuracy separately, thereby recovering the lost record-level importance information while maintaining manageable evaluation complexity.
Solution Approach 2:
The patent introduces clustered data groups as an intermediary between individual records and the predictive model. Rather than directly evaluating each individual record's importance (which would be computationally prohibitive) or evaluating features independently (which loses record-level information), the method uses clustered groups as a mediator. Each group's importance is evaluated by comparing model accuracy with and without that group, providing a scalable approach to record-level importance analysis.
2Reliability
If all training records are used uniformly in model building, then comprehensive data coverage is achieved, but noise and harmful records can negatively impact model accuracy
Solution Approach 1:
The patent applies local quality by evaluating and treating different data groups differently based on their specific characteristics and impact on model accuracy. Instead of uniformly including all records, the method assigns different importance levels to different clustered groups (e.g., high importance for beneficial groups, low or zero importance for harmful/noisy groups). This differential treatment based on local group quality enables the model to benefit from comprehensive data coverage while filtering out noise and harmful records that would otherwise degrade performance.
Solution Approach 2:
The patent inverts the conventional approach by instead of trying to identify and remove harmful records directly, it evaluates the positive contribution of each clustered group to model accuracy. By measuring how much model accuracy decreases when a group is removed, the method identifies beneficial groups to retain. Groups whose removal causes minimal or negative accuracy change are identified as harmful or noisy and are effectively filtered out through this inverse evaluation approach.
3Measurement precision
If individual records are evaluated for importance, then precise record-level insights are obtained, but computational complexity becomes prohibitive for large datasets
Solution Approach 1:
The patent merges multiple individual records into clustered data groups based on feature similarities and patterns. By combining records within each group that share common characteristics, the method reduces the number of evaluations needed from individual record level to group level. This merging maintains measurement precision at the group level while dramatically reducing computational complexity, making record-level importance analysis feasible for large datasets.
Solution Approach 2:
The patent applies partial action by evaluating only representative clustered groups rather than every single individual record. Instead of performing exhaustive evaluation on all records (excessive action), the method selectively evaluates clustered groups that capture the essential variations in the data. This partial evaluation approach achieves sufficient measurement precision for identifying important records while avoiding the prohibitive computational cost of evaluating every individual record.
Data Source
AI summary
Embodiments herein describe computer-implemented methods, computer program products, and computer systems for determining record importance. The methods may include providing a first trained model having a first trained model accuracy. Further, the methods may include clustering the training data to generate clustered data groups, extracting a first clustered data group from the clustered data groups to identify first model test data, processing the first model test data using the first trained model to generate first trained model output data having first test data accuracy, and labeling the first clustered data group with a first record importance level based on a first comparison between the first trained model accuracy and the first test data accuracy. Further, the methods may include clustering the training data by processing the training data using hierarchical clustering to group the training data into the clustered data groups based on features corresponding to a hierarchy of importance.