Shallow Neural Network Feature Selection and Interaction Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analysis techniques face challenges in feature selection and feature interaction learning, particularly in large-scale datasets, due to computational intensity and reliance on low-fidelity operations, which limits the ability to capture higher-order interactions effectively.
Innovation Solution
A computing system that employs a shallow neural network for concurrent feature selection and feature interaction learning, enabling non-linear interactions of arbitrary order without exhaustive search, and is scalable to large datasets with modest compute resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exhaustive pairwise interaction learning is performed, then feature interaction accuracy is improved, but computational complexity increases exponentially
Solution Approach 1:
The patent replaces traditional mechanical/computational methods of calculating feature interactions with a neural network-based system. The neural network automatically learns feature interactions through its architecture and training process, substituting the exhaustive computational approach with a learning-based approach that scales more efficiently.
Solution Approach 2:
The patent changes the parameters of the neural network (architecture, learning rate, regularization) to optimize the balance between capturing higher-order feature interactions and maintaining computational efficiency. By adjusting these parameters, the system can learn complex interactions without requiring exhaustive computation.
2Reliability
If higher-order feature interaction learning is performed, then data analysis quality is improved, but computational resources required increase exponentially
Solution Approach 1:
The patent applies partial action by focusing the neural network's learning capacity on the most significant feature interactions rather than computing all possible interactions. The network learns to attend to and prioritize relevant interactions, achieving high-quality data analysis without the exponential computational cost of exhaustive higher-order interaction learning.
Solution Approach 2:
The neural network acts as an intermediary between raw features and final predictions, automatically discovering and learning feature interactions at various orders. This intermediary layer transforms the input features through multiple non-linear transformations, capturing complex interactions efficiently without direct computation of all interaction terms.
3Productivity
If linear operations are used for feature interaction approximation, then computational efficiency is improved, but interaction learning fidelity deteriorates
Solution Approach 1:
The patent introduces non-linearity into the feature interaction learning process through the neural network's activation functions. These non-linear transformations (curved decision boundaries) allow the system to capture complex, non-linear feature interactions while maintaining computational efficiency through vectorized operations and optimized training algorithms.
Solution Approach 2:
The patent combines multiple neural network layers with different transformation capabilities to create a composite processing system. Each layer contributes different types of feature transformations and interactions, and their composition enables the system to achieve high interaction learning fidelity while maintaining computational efficiency through the synergistic effect of multiple specialized components.
Data Source
AI summary
Data analysis and neural network training technology includes generates, based on a sparse neural network, a feature selection ranking representing a ranked list of features from input data, where the sparse neural network is a shallow neural network trained with the input data and then pruned, generates, based on the sparse neural network, a feature set dictionary representing interactions among features from the input data, and performs, based on the feature selection ranking and the feature set dictionary, one or more of generating an output analysis of insights from the input data and the sparse neural network, or training of a second neural network. The technology can also adjust the input data based on the feature set ranking to produce adjusted input data, where the sparse neural network is re-trained based on the adjusted input data and then pruned prior to generating the feature set dictionary.