Interactive Feature Generation via Edge Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic interactive feature generation methods face challenges due to large search spaces and lack of interpretability, making it difficult to generate useful and explicit interactive features in a simple and effective training manner.
Innovation Solution
An interactive feature generation system that uses a feature graph to model features as nodes and interactions as edges, employing a Markov Decision Process for edge search and differentiable optimization to iteratively generate higher-order features, and trains a lightweight model for real-time inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If search-based methods are used for automatic interactive feature generation, then automation is improved, but the search space becomes excessively large leading to inefficiency
Solution Approach 1:
The patent segments the feature generation process into distinct orders (first-order features, second-order interactions, third-order interactions, etc.). Each order is generated separately and systematically, dividing the overwhelming search space into manageable segments that can be processed efficiently in sequence rather than attempting to search all possible feature combinations simultaneously.
Solution Approach 2:
The patent performs preliminary action by generating and evaluating lower-order features before proceeding to higher-order interactions. This systematic progression from first-order to second-order to third-order features allows the system to build upon previously identified useful features, avoiding redundant searches and focusing computational effort on promising feature combinations.
2Extent of automation
If deep-learning-based methods are used for automatic interactive feature generation, then automation is improved, but interpretability is lost due to implicit nature of feature interactions
Solution Approach 1:
The patent incorporates feedback mechanisms where the performance of generated features is evaluated and used to guide subsequent feature generation. The system uses performance metrics from lower-order features to inform the generation of higher-order features, creating a feedback loop that maintains interpretability by systematically tracking which feature combinations are useful and why, rather than relying on black-box deep learning interpretations.
Solution Approach 2:
The system performs self-service by automatically generating and evaluating feature interactions while maintaining explicit representations of these interactions. The feature generation process itself serves to identify and document the interpretability information, with each generated feature explicitly representing a specific interaction pattern that can be traced back to its component features and evaluated for its contribution to model performance.
3Loss of information
If traditional expert-based methods are used for feature interaction generation, then interpretability is maintained, but the process becomes time consuming and task-specific
Solution Approach 1:
The patent implements a universal feature generation framework that can automatically generate interpretable features across different tasks and domains. The systematic approach to generating features of various orders applies universally to different machine learning problems without requiring task-specific customization, thereby maintaining interpretability while achieving high productivity through automation that works across multiple applications.
Data Source
AI summary
An interactive feature generation system may receive a plurality of distinct features that are associated with an application, and associate a plurality of nodes in a feature graph of a first order to the plurality of distinct features. The interactive feature generation system may iteratively generate interactive features of a higher order from interactive features of a lower order to form a plurality of feature graphs of different orders. The interactive feature generation system may then propagate respective interactive features of the plurality of feature graphs of the different orders to a neural network to determine a number of interactive features of one or more orders, the determined number of interactive features of the one or more orders being used for training a predictive model to make inferences for the application.