Neural Network Feature Engineering via Transitive Closure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for configuring neural networks to make predictions from large datasets face challenges due to the complexity and resource requirements of feature crossing, leading to sparse matrices and inefficient resource utilization, with no systematic approach for selecting feature crosses to achieve a dense matrix without compromising accuracy or data granularity.
Innovation Solution
The method constructs a transitive closure data structure to identify essential paths between features, reduces it to form a reduced ontology graph, and computes upper and lower bounds for feature crosses to determine optimal layer sizes, transforming the dataset into a sparse matrix while maintaining accuracy, and performing forward materialization to build an ontology graph for efficient feature engineering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature crossing is performed to create comprehensive feature combinations, then prediction accuracy is improved, but device complexity and resource requirements increase significantly
Solution Approach 1:
The patent extracts and selects only the most essential feature crosses from the complete set of possible combinations. By using transitive closure to identify necessary paths between features and removing redundant crossings, the system retains only the minimal set of feature combinations needed for accurate predictions, thereby reducing neural network complexity while maintaining prediction accuracy.
Solution Approach 2:
The patent changes the parameter of feature cross selection from exhaustive combinations to a reduced set based on transitive closure analysis. This parameter change transforms the approach from creating all possible feature combinations to selecting only those that are necessary for maintaining predictive accuracy, thus reducing computational complexity.
2Loss of information
If all possible feature crosses are generated, then data granularity is preserved, but matrix sparsity increases and resource utilization becomes inefficient
Solution Approach 1:
The patent extracts only the essential feature crosses that maintain data granularity through transitive closure analysis. By removing redundant feature combinations that do not contribute to preserving information relationships, the system maintains data granularity while significantly reducing the number of features processed, thereby improving computational resource utilization efficiency.
Solution Approach 2:
The patent discards redundant feature crosses that do not contribute to maintaining information relationships, while recovering and preserving only the essential feature combinations through transitive closure. This selective discarding reduces computational load while the recovery of essential features maintains data granularity.
3Measurement precision
If neural network layer size is increased to represent more features, then prediction accuracy is improved, but computational resources and training time increase
Solution Approach 1:
The patent extracts and selects only the minimal set of essential feature crosses needed for accurate predictions through transitive closure analysis. This extraction reduces the number of features that must be represented in neural network layers, thereby reducing computational resource consumption and training time while maintaining prediction accuracy.
Solution Approach 2:
The patent changes the parameter of layer size from representing all possible features to representing only the essential features identified through transitive closure. This parameter optimization reduces computational resource consumption while preserving the ability to make accurate predictions.
Data Source
AI summary
A transitive closure data structure is constructed for a pair of features represented in a vector space corresponding to an input dataset. The data structure includes a set of entries corresponding to a set of all possible paths between a first feature in the pair and a second feature in the pair in a graph of the vector space. The data structure is reduced by removing a subset of the set of entries such that only a single entry corresponding to a single path remains in the transitive closure data structure. A feature cross is formed from a cluster of features remaining in a reduced ontology graph resulting from the reducing the transitive closure data structure. A layer is configured in a neural network to represent the feature cross, which causes the neural network to produce a prediction that is within a defined accuracy relative to the dataset.