One Permutation Hashing and Densification for Trait Recommendations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital content distribution systems face inefficiencies, inflexibilities, and inaccuracies due to computationally expensive processes, rigid similarity determination based on overlap, and high variance in recommendations, leading to inefficient resource usage and inaccurate trait similarity predictions.
Innovation Solution
The system employs one permutation hashing to generate sketches of traits and a populated-value-slot-based densification routine to efficiently fill empty bins, maintaining locality sensitive hashing properties, and combines sketches to generate a recommendation model that flexibly identifies similar traits using trait embeddings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems employ models that analyze every user associated with every trait to determine overlap, then similarity determination is based on complete population overlap, but computational resources (time, processing power, memory) are excessively consumed
Solution Approach 1:
The patent segments the complete user population into smaller hash buckets using hashing functions. Instead of analyzing every user across all traits, the system divides users into discrete buckets and only compares users within the same bucket, dramatically reducing the computational scope while maintaining similarity measurement capability through the hash-based segmentation structure
Solution Approach 2:
The patent creates hash-based representations (sketches) of user populations that serve as compressed copies of the complete data. These sketches capture essential population characteristics in a condensed form, allowing similarity comparison without processing the full user datasets, thus reducing computational resources while preserving measurement accuracy
2Measurement precision
If systems use multiple steps of raw data analysis to generate sketches (e.g., minwise hashing), then sketch accuracy is improved, but training computational cost increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for similarity measurement into compact hash sketches, discarding redundant data processing steps. By taking out only the critical hash-based representations and eliminating unnecessary intermediate analysis steps, the system maintains sketch accuracy while dramatically reducing training time and computational overhead
3Manufacturing precision
If densification routines iteratively analyze unpopulated value slots to fill them, then sketch completeness is improved, but matching time increases significantly especially when populated slots are low
Solution Approach 1:
The patent inverts the traditional densification approach by not actively searching for populated slots to fill unpopulated ones. Instead, it uses hash functions to directly populate slots in a single pass, eliminating the iterative matching process and reducing densification time from potentially quadratic to linear complexity while maintaining sketch completeness
4Device complexity
If models determine similarity strictly based on overlap of trait populations, then computational process is simplified, but prediction accuracy of relevant traits decreases
Solution Approach 1:
The patent changes the parameter for similarity determination from strict population overlap to hash-based sketch comparison. By transforming the similarity metric into a hash space where sketches are compared, the system maintains relatively simple computational processes while significantly improving trait similarity prediction accuracy, as the hash-based approach captures nuanced population characteristics better than simple overlap metrics
Data Source
AI summary
The present disclosure relates to training a recommendation model to generate trait recommendations using one permutation hashing and populated-value-slot-based densification. In particular, the disclosed systems can train the recommendation model by computing sketch vectors corresponding to traits using one permutation hashing. The disclosed systems can then fill in unpopulated value slots of the sketch vectors using populated-value-slot-based densification. The disclosed systems can combine the resulting densified sketches to generate the trained recommendation model. For example, in some embodiments, the disclosed systems can combine the sketches by generating a plurality of locality sensitive hashing tables based on the sketches. In some embodiments, the disclosed systems generate a count sketch matrix based on the sketches and generate trait embeddings based on the count sketch matrix using spectral embedding. Based on the trait embeddings, the disclosed systems can utilize the recommendation model to flexibly and accurately determine the similarity between traits.