Feature Engineering via Landmark-Based Distance Matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-dimensional datasets often include irrelevant features, making feature engineering challenging and computationally expensive, especially in machine learning applications like text mining and speech recognition, where selecting optimal features is crucial for model performance.
Innovation Solution
A method that involves randomly selecting landmark observation vectors, computing neighbor observation vectors, and projecting data into a neighborhood subspace to create a feature distance matrix, which is used to train a machine learning model for improved classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If feature engineering is performed on high-dimensional datasets, then model performance is improved, but computational cost and complexity increase
Solution Approach 1:
The patent segments the high-dimensional feature space by selecting landmark observation vectors and dividing remaining vectors into neighborhoods based on distance thresholds. This segmentation transforms the complex global feature engineering problem into multiple simpler local processing tasks, reducing computational complexity while maintaining model performance.
Solution Approach 2:
The patent extracts and removes irrelevant features by computing distance metrics between observation vectors and landmark vectors. Features with distances exceeding predefined thresholds are identified as irrelevant and excluded from the final feature representation, thereby improving model performance without the computational burden of processing all original features.
2Loss of information
If all features in high-dimensional datasets are used, then comprehensive data representation is achieved, but irrelevant features increase computational expense
Solution Approach 1:
The patent applies local quality by treating different regions of the feature space differently. Landmark observation vectors and their nearby neighbors are processed with full detail, while distant vectors are approximated or excluded. This localized approach maintains data representation completeness for relevant features while reducing computational expense by ignoring irrelevant distant features.
Solution Approach 2:
The patent performs partial action by processing only a subset of observation vectors (landmarks and their nearby neighbors) rather than all vectors in the high-dimensional dataset. This partial processing achieves sufficient data representation for model training while significantly reducing computational expense compared to exhaustive feature analysis.
3Reliability
If domain specific feature engineering methods are used, then promising results are achieved, but the methods are challenging and computationally expensive
Solution Approach 1:
The patent changes parameters by using distance metrics (e.g., Euclidean distance) as a universal measure across different domains, replacing complex domain-specific feature engineering methods. By transforming the problem into a distance-based clustering and selection framework, it achieves promising classification results with reduced analytical complexity and computational expense.
Data Source
AI summary
A computing device determines a sparse feature representation for a machine learning model. Landmark observation vectors are randomly selected. Neighbor observation vectors are randomly selected that are less than a predefined distance from a selected landmark observation vector. The observation vectors are projected into a neighborhood subspace defined by principal components computed for the neighbor observation vectors. A distance vector includes a distance value computed between each landmark observation vector and each observation vector of the projected observation vectors. Nearest landmark observation vectors are selected from the landmark observation vectors for each observation vector. A second distance vector that includes a second distance value computed between each observation vector and each landmark observation vector is added to a feature distance matrix, where the second distance value is zero for each landmark observation vector not included in the nearest landmark observation vectors. A model is trained using the feature distance matrix.