Feedback-Assisted Feature Selection for Higher-Order Interactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional feature-selection techniques in machine learning struggle to identify higher-order interactions between features, often masking these interactions with lower-order effects, leading to suboptimal model performance and increased training time.
Innovation Solution
Utilizing quantum annealing-based quadratic unconstrained binary optimization (QUBO) models to solve the feature-selection problem, enabling the identification of higher-order feature interactions through quantum computing systems, which include multivariate effect, multi-term mutual information, ensemble, and feedback-assisted optimization models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional feature-selection techniques are used, then the feature selection process is simpler and faster to execute, but higher-order interactions between features are masked and not identified
Solution Approach 1:
The patent transforms the feature selection problem from traditional univariate or bivariate analysis into a multivariate optimization problem using QUBO models. This dimensional transformation allows simultaneous evaluation of multiple feature interactions (higher-order interactions) by mapping feature combinations to quantum states, enabling detection of interactions that traditional sequential methods mask.
Solution Approach 2:
The patent replaces classical computational mechanics (traditional feature selection algorithms) with quantum computing mechanics (quantum annealing). This substitution enables parallel evaluation of exponential feature combinations through quantum superposition and entanglement, uncovering higher-order interactions that classical methods cannot detect due to computational complexity constraints.
2Reliability
If comprehensive feature combinations are evaluated to identify higher-order interactions, then model accuracy improves, but training time increases significantly
Solution Approach 1:
The patent performs preliminary quantum annealing optimization to identify the optimal feature subset before model training. By pre-selecting features using quantum parallel evaluation of all combinations, the method avoids time-consuming iterative training of models with different feature sets, significantly reducing total computational time while maintaining high accuracy.
Solution Approach 2:
The patent uses quantum copying of feature combination states through quantum superposition, allowing simultaneous evaluation of exponentially many feature subsets. This copying mechanism enables comprehensive feature interaction analysis without the sequential time cost of classical brute-force evaluation, achieving both high accuracy and efficiency.
3Adaptability or versatility
If more features are included in the model to capture interactions, then model completeness improves, but overfitting increases and training efficiency decreases
Solution Approach 1:
The patent extracts only the essential features and their interactions that contribute most to predictive accuracy, as identified by the quantum optimization process. By removing redundant and irrelevant features through quantum-evaluated feature importance metrics, the method achieves model completeness with minimal necessary features, preventing overfitting and improving training efficiency.
Solution Approach 2:
The patent changes the optimization parameters from traditional accuracy-only metrics to a composite objective function that balances accuracy, feature interaction capture, and model sparsity. This parameter transformation guides the quantum annealer to find feature subsets that achieve completeness while maintaining efficiency by penalizing excessive feature inclusion.
Data Source
AI summary
Techniques are disclosed relating to feature selection based on feedback-assisted optimization models. In various embodiments, for example, the disclosed techniques include accessing a training dataset that includes a plurality of data samples that include data values for a plurality of features, and a set of labels corresponding to the plurality of data samples. In some embodiments, a computer system performs feature-selection operations to select, from the plurality of features, a subset of features to include in a reduced feature set. For example, in some embodiments the feature-selection operations include processing the training dataset based on an optimization model, where an objective function utilized in the optimization model utilizes performance feedback information corresponding to machine learning models that are trained based on candidate feature sets. Based on the feature-selection operation, the computer system may generate an output value that indicates the subset of features to include in the reduced feature set.