Locality Sensitive Hashing for Embedding Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face high computational costs and resource expenditure when introducing new vocabulary, as machine learning models need to be retrained to generate and accommodate new embeddings, making it inefficient to use new vocabulary effectively.
Innovation Solution
The system generates hashes for existing vocabulary words and uses locality sensitive hashing techniques to determine the most similar existing word for new vocabulary, allowing the use of existing embeddings instead of generating new ones, thereby avoiding the need for model retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If new vocabulary words are added to the set, then the system can process more content items and expand its knowledge base, but the computational cost and time required for model retraining increases significantly
Solution Approach 1:
The patent uses locality sensitive hashing to generate hash values for new vocabulary words and finds existing words with similar hash values. The embedding of the similar existing word is copied and assigned to the new word, avoiding the need to generate a new embedding through expensive model retraining. This copying approach allows the system to expand vocabulary coverage while minimizing computational resource consumption.
2Measurement precision
If model retraining is performed to generate embeddings for new vocabulary, then the embeddings accurately represent the new words, but the time and computational resources required increase prohibitively
Solution Approach 1:
The patent pre-computes hash values for all vocabulary words during an initial setup phase. When new vocabulary is added, the system can immediately compute hashes and find similar words without performing time-consuming model retraining. This preliminary action of pre-computing hashes enables rapid adaptation to new vocabulary while maintaining reasonable embedding accuracy through similarity-based assignment.
3Productivity
If existing embeddings are reused for new vocabulary words, then computational resources are saved, but the system may lose precision in representing the new words
Solution Approach 1:
The patent introduces locality sensitive hashing as an intermediary mechanism between new vocabulary words and existing embeddings. The hash values serve as a bridge that enables the system to find and select the most appropriate existing embedding for each new word based on similarity, rather than randomly assigning or always retraining. This intermediary approach balances efficiency with reasonable precision.
Data Source
AI summary
In some embodiments, a computing system may generate a prediction related to a new category (not included in a set of categories) using a machine learning model trained on a set of embeddings corresponding to the category set. As an example, the computing system may generate a set of hashes such that each hash of the hash set is mapped to an embedding of the embedding set. When a new category added to the category set, the computing system may generate a given hash for the new category and identify a first hash of the hash set that matches the given hash. Based on identifying the first hash as a matching hash, the computing system may use an existing embedding (e.g., mapped to the first hash) with the machine learning model in connection with the new category, thereby avoiding a need to add a new embedding to the embedding set.