Correlation Matrix Embeddings via Sparse Networks for Small-Data ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning approaches face challenges in effectively capturing relationships between nodes in networks due to high cardinality nominal categorical variables, leading to increased feature space and reduced predictive power, especially in small data problems, and traditional one-hot encoded representations fail to consider similarity between entities.
Innovation Solution
A machine learning framework that generates n-dimensional vector representations (embeddings) using neural networks to capture relationships between assets, transforming correlation matrices into distance matrices, creating sparse networks, and applying algorithms like Node2Vec and Word2Vec to learn embeddings, with an evaluation module for tuning hyperparameters and assessing embeddings using V-measure, similarity, and analogical inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional one-hot encoded representation is used for high cardinality nominal categorical variables, then the feature space increases, but the predictive power reduces especially in small data problems
Solution Approach 1:
The patent transforms high-cardinality nominal categorical variables from traditional one-hot encoded representation into n-dimensional vector embeddings. This dimensionality transformation allows the model to capture relationships between entities in a continuous vector space, reducing the effective feature space while preserving and even enhancing predictive power through learned semantic relationships.
Solution Approach 2:
The patent changes the representation parameters of categorical variables from discrete one-hot vectors to continuous n-dimensional embeddings. By learning optimal embedding dimensions and transformations through neural networks, the system adapts the feature representation to better suit the specific data characteristics and downstream tasks, improving predictive performance.
2Device complexity
If traditional one-hot encoded representation is used, then each entity representation is independent, but similarity between entities is not considered
Solution Approach 1:
The patent merges the representation of similar entities by mapping them to nearby positions in the n-dimensional embedding space. Entities with similar characteristics are positioned closer together in the vector space, allowing the model to automatically capture and utilize similarity information without explicit similarity calculations.
Solution Approach 2:
The patent introduces n-dimensional vector embeddings as an intermediary representation between the original categorical variables and the downstream model. This intermediary layer captures similarity relationships through the geometric structure of the embedding space, bridging the gap between discrete categories and continuous similarity measures.
Data Source
AI summary
System, method, and a computer program product for generating embeddings are provided. A machine learning framework generates a fully connected network from a dataset associated with words. The words correspond to nodes in the fully connected network. The weights are associated with correlations between the nodes and correspond to the links in the fully connected network. The machine learning framework transforms the correlations corresponding to the links into distances. The machine learning framework generates a sparse network from the fully connected network based on the distances. From the sparse network, machine learning framework determines sentence structures by traversing the nodes. Using the sentence structures, the machine learning framework uses a neural network to generate embeddings in the embedded space.