Compact Entity Identifier Embeddings via Hash Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for using entity IDs in machine learning models face inefficiencies due to the high dimensionality and sparsity of IDs, requiring significant computational and storage resources for one-hot encoding and one-to-one mapping, which becomes impractical with large numbers of entities.
Innovation Solution
The use of hash functions to convert entity IDs into hash values, which are then used as indices for lookup tables to generate intermediate vectors, aggregated into compact embeddings, allowing for efficient representation and updating of entity relationships and interactions within machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If one-hot encoding or one-to-one mapping is used to represent entity IDs, then the representation is complete and accurate, but the computational and storage resources required become prohibitively large for high-dimensional IDs
Solution Approach 1:
The patent segments the high-dimensional entity ID space into multiple lower-dimensional hash tables. Each hash function maps entity IDs to specific hash tables, dividing the large-scale representation problem into multiple smaller, manageable components that require less computational and storage resources individually
Solution Approach 2:
The patent transforms the high-dimensional entity ID representation problem into a multi-dimensional solution by using multiple hash tables with different dimensions. Instead of representing all entities in a single high-dimensional space, the system distributes entities across multiple lower-dimensional hash tables, reducing the dimensional burden on any single representation
2Productivity
If hash functions are used to compress entity IDs into lower-dimensional embeddings, then computational efficiency improves, but information loss may occur due to dimensionality reduction
Solution Approach 1:
The patent merges multiple intermediate vector representations from different hash tables through element-wise aggregation to create the final embedding. This combining approach consolidates information from multiple sources, compensating for potential information loss in individual hash tables and preserving more complete entity relationship information
Solution Approach 2:
The patent introduces intermediate vectors as mediator representations between the original entity IDs and the final embeddings. These intermediate vectors serve as temporary carriers that preserve relationship information during the dimensionality reduction process, allowing information to be maintained and then aggregated in the final embedding stage
3Adaptability or versatility
If multiple hash functions and lookup tables are used to generate embeddings, then the representation capacity increases, but the device complexity increases
Solution Approach 1:
The patent makes the hash function system universal by designing hash functions and lookup tables that can handle multiple types of entity IDs (user IDs, item IDs, advertiser IDs, etc.) through a unified framework. This multi-functional design increases representation capacity while avoiding the need for separate complex systems for each entity type
Data Source
AI summary
The disclosed embodiments provide a system for processing data. During operation, the system applies a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values. Next, the system produces a first set of intermediate vectors from the first set of hash values and a first set of lookup tables by matching each hash value in the first set of hash values to an entry in a corresponding lookup table in the first set of lookup tables. The system then performs an element-wise aggregation of the first set of intermediate vectors to produce a first embedding. Finally, the system outputs the first embedding for use by a machine learning model.