Batched Feature Significance Identification for ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning solutions lack efficiency in identifying significant input features and providing explainability for model outputs, especially when dealing with large batches of predictions, as existing methods are time-consuming and costly, and do not scale well for production systems.
Innovation Solution
A computer-implemented method that slices scoring payload data into batches, generates a correlation coefficient matrix for each input feature, identifies significant features based on correlation coefficient values, and performs action steps to enhance prediction accuracy and model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If correlation analysis is performed on large batches of predictions, then identification accuracy of significant features is improved, but processing time and computational cost increase significantly
Solution Approach 1:
The patent divides the large batch of predictions into multiple smaller batches and processes them in parallel. Each smaller batch undergoes correlation analysis independently, and the results are aggregated to identify significant features. This segmentation reduces the computational burden on any single processing unit while maintaining overall identification accuracy through result aggregation.
2Measurement precision
If correlation analysis is performed on large batches of predictions, then identification accuracy of significant features is improved, but computational resources required increase
Solution Approach 1:
The patent segments the computational workload by dividing predictions into smaller batches that can be processed in parallel. This distribution across multiple processing units reduces the computational resource burden on any single unit while collectively achieving the same identification accuracy as a monolithic approach would provide.
3Ease of manufacture
If existing methods are used for feature identification, then implementation is simpler, but scalability to production systems is poor
Solution Approach 1:
The patent implements a segmented batch processing architecture that maintains implementation simplicity through modular design while achieving scalability. Each batch is processed independently using the same correlation analysis logic, allowing the system to scale horizontally by adding more processing units without complicating the core algorithm.
Data Source
AI summary
Identifying input feature significance for a machine learning model is provided. A set of scoring payload data corresponding to a set of input features of the machine learning model is sliced into a predefined number of batches. Using the sliced set of scoring payload data, a correlation coefficient matrix for each respective input feature of the machine learning model is generated based on input of each particular input feature into the machine learning model and a corresponding output from the machine learning model for each particular input feature. A correlation coefficient value is extracted from the correlation coefficient matrix for each particular input feature and the corresponding output from the machine learning model. A set of significant input features having a corresponding correlation coefficient value greater than a predefined correlation coefficient threshold level is identified. A set of action steps is performed regarding the set of significant input features.