Ranked Factor Selection for ML Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning models face challenges in handling large datasets with thousands of factors, leading to insufficient memory issues and sub-optimal performance, particularly in the insurance industry, where manual examination of correlations and interactions is not feasible.
Innovation Solution
A systematic approach to automatically select a small number of relevant factors from a dataset by determining their correlation with a target prediction, using a trained ML model, and ranking them based on their scores, thereby reducing the computational cost and improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional ML models process large datasets with thousands of factors, then comprehensive analysis coverage is improved, but memory capacity is exceeded and computational cost increases
Solution Approach 1:
The patent extracts and removes irrelevant or less important factors from the dataset, keeping only the most relevant ones for model training. This is achieved through automated feature selection techniques that identify and eliminate redundant features, thereby reducing memory requirements while maintaining model performance.
Solution Approach 2:
The patent transforms the dataset by changing parameters such as factor selection criteria, data sampling strategies, and feature engineering approaches. This allows the model to work efficiently with a reduced set of factors while maintaining or improving predictive accuracy through optimized parameter configurations.
2Loss of information
If manual examination of correlations and interactions is performed, then understanding of data relationships is improved, but time consumption increases
Solution Approach 1:
The patent implements automated feature selection and correlation analysis systems that perform data relationship examination without human intervention. The system automatically identifies correlations, interactions, and patterns in the data, eliminating the need for manual analysis while preserving comprehensive understanding of data relationships.
Solution Approach 2:
The patent replaces manual mechanical examination with computational algorithms and automated analysis tools. Machine learning models and statistical algorithms automatically detect correlations and interactions, substituting human manual analysis with efficient computational processes that maintain or improve understanding while dramatically reducing time requirements.
3Loss of information
If all factors are considered in ML modeling, then prediction completeness is improved, but computational cost increases
Solution Approach 1:
The patent extracts only the most relevant factors for prediction, removing redundant and less important features. This selective extraction maintains prediction completeness by retaining critical information while eliminating computational waste associated with processing unnecessary factors.
Solution Approach 2:
The patent applies partial action by considering only the necessary subset of factors required for accurate prediction, rather than processing all available factors. This approach achieves sufficient prediction completeness with reduced computational effort by focusing on the most impactful features.
Data Source
AI summary
Described herein are techniques to a systematic approach to reduce the number of factors of an input dataset that impact a target prediction of a trained ML model. The techniques include obtaining a dataset of typed data points and ascertaining the factors of the data points based, at least in part, on the datatypes of the data points. The techniques also include obtaining an indicator of correlation of each factor ascertained in the dataset to a target prediction by a trained ML model and assigning a score to each respective factor ascertained in the dataset based on the indicator of correlation of each factor. The techniques further include ranking the factors ascertained in the dataset based on the score of each factor, selecting factors from the factors ascertained in the dataset, and providing the selected factors for making the target prediction by the trained ML model.