Predictive Analytics Feature Extraction for Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning frameworks face difficulties in training efficiency and accuracy when dealing with highly sparse and high-cardinality categorical data, leading to increased complexity and dimensionality issues, making it challenging to generate accurate predictions.
Innovation Solution
The method involves processing sparse and high-cardinality categorical data by extracting low-order and high-order feature values through feature engineering transformations and attention-based feature selection, reducing data sparsity and cardinality, and merging these features to improve training efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional machine learning frameworks are used to process highly sparse and high-cardinality categorical data, then the framework can handle the data in its original form, but training efficiency and accuracy deteriorate due to increased complexity and dimensionality
Solution Approach 1:
The patent extracts low-order features from high-cardinality categorical data by identifying and selecting a subset of the most relevant feature values. This extraction process removes the complexity and sparsity issues from the original data while preserving the essential information needed for accurate predictions, directly resolving the contradiction between handling original data and maintaining training efficiency
Solution Approach 2:
The patent transforms the data representation by changing parameters such as cardinality and sparsity through feature extraction. By converting high-cardinality categorical data into low-order feature representations with reduced cardinality, the system maintains prediction accuracy while significantly reducing data complexity and improving training efficiency
2Productivity
If feature engineering transformations are performed to reduce data sparsity and cardinality, then training efficiency improves, but the complexity of the processing increases
Solution Approach 1:
The patent segments the feature engineering process into distinct iterative steps: identifying input feature values, generating candidate low-order features, scoring these features, and selecting the best ones. This segmentation makes the complex processing manageable and systematic, improving training efficiency without overwhelming complexity
Solution Approach 2:
The patent performs partial feature engineering by extracting only the necessary low-order features rather than processing all possible feature combinations. By selecting a subset of the most important features through scoring and selection mechanisms, the system achieves improved training efficiency with controlled processing complexity
3Reliability
If low-order feature extraction is performed through iterative feature engineering, then training accuracy improves, but the processing time increases
Solution Approach 1:
The patent uses periodic iterative feature engineering where features are extracted and evaluated in cycles. Each iteration refines the feature set by adding the most valuable low-order features while stopping when performance plateaus, thus improving training accuracy without excessive processing time
Solution Approach 2:
The patent incorporates feedback mechanisms where each extracted low-order feature is scored and evaluated for its contribution to prediction accuracy. This feedback guides the iterative process to focus on features that actually improve accuracy, preventing wasted processing time on irrelevant features while maintaining high training accuracy
Data Source
AI summary
There is a need for solutions that predictive data analytics with improved training efficiency and/or accuracy. This need can be addressed by, for example, processing an original feature entry to generate a group of low-order feature values, including performing a first number of iterations of a feature engineering transformation to generate a group of engineered feature values and determining the group of low-ordered feature values based on a number of feature values from the group of engineered feature values; processing the original feature entry to generate a group of high-order feature values; merging the group of low-order feature values and the group of high-order feature values to generate a processed feature entry corresponding to the original feature entry; and providing the processed feature entry as an input to a prediction unit.