Record Embeddings for Structured Records via Token Vocabulary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional programming techniques face challenges in implementing tasks like classification and regression with structured records, as they require defining features explicitly, and machine learning methods like neural networks become computationally expensive with increasing dataset dimensionality, especially when dealing with sparse and long one-hot encoded vectors.
Innovation Solution
The method generates record embeddings by preprocessing structured records to create a token vocabulary, initializing and updating token embeddings using a neural network, and forming record embeddings through attribute embeddings, which are more suitable for matching, regression, and classification tasks, and can be used across various domains without labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If one-hot encoded vectors or TF-IDF weighted vectors are used to represent structured records, then the records can be processed by machine learning algorithms, but the vectors are sparse and long, making them computationally expensive
Solution Approach 1:
The patent transforms the representation parameters of structured records from sparse high-dimensional one-hot encoded vectors or TF-IDF weighted vectors to dense low-dimensional continuous vectors through neural network embedding layers. This parameter transformation maintains the ability to represent categorical data while dramatically reducing dimensionality and sparsity, thereby improving computational efficiency for machine learning tasks
Solution Approach 2:
The patent creates simplified copies of the original structured record data in the form of embedding vectors. These embedding vectors are continuous approximations that capture the essential semantic information of the original sparse vectors but in a much more compact and computationally efficient format, enabling faster processing without significant loss of information
2Ease of manufacture
If traditional programming techniques are used for classification and regression tasks, then the implementation is straightforward with explicit feature definitions, but it becomes difficult to handle complex patterns in structured records
Solution Approach 1:
The patent introduces embedding vectors as an intermediary representation layer between the original structured records and the machine learning models. This intermediary transformation automatically learns optimal feature representations from the data, bridging the gap between simple traditional programming approaches and the need to capture complex patterns, thereby enabling both ease of implementation and high adaptability
Solution Approach 2:
The patent replaces manual mechanical feature engineering and explicit feature definition with automatic neural network-based embedding learning. Instead of programmers manually defining features for comparison, the system automatically learns meaningful representations through the embedding process, substituting manual mechanical operations with automated learning mechanisms
3Adaptability or versatility
If machine learning techniques like neural networks are employed for classification and regression tasks, then complex patterns can be captured, but the training time and computing requirements increase with dataset dimensionality
Solution Approach 1:
The patent segments the overall machine learning process into two distinct stages: (1) an offline embedding generation stage that pre-processes structured records into embedding vectors, and (2) an online prediction stage that uses these pre-generated embeddings for fast inference. This segmentation allows computationally intensive pattern learning to occur once during embedding generation, while subsequent predictions benefit from reduced dimensionality and faster processing
Solution Approach 2:
The patent performs preliminary action by pre-computing embedding vectors for structured records before the actual classification or regression tasks. These pre-computed embeddings capture complex patterns in advance, so that during actual model training and inference, the system works with already-processed low-dimensional vectors rather than raw high-dimensional data, significantly reducing training time and computational requirements
Data Source
AI summary
Techniques for generating record embeddings from structured records are described. A record embeddings generating engine processes structured records to build a token vocabulary. Token embeddings are created for each token in the vocabulary. The token embeddings are trained using a loss function that relates the token embeddings to the record-attribute-data structure of the structured records. A record embedding is assembled from the trained token embeddings.