Nearest Neighbor Matching in Shared Embedding Space
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems for targeting communications to users are computationally expensive and inefficient due to high memory overhead when calculating nearest neighbors in a common embedding space, especially when dealing with a large number of users.
Innovation Solution
The system applies a condition to cull a set of users before determining nearest neighbors to an entity in a common embedding space, reducing computational complexity and memory overhead by discarding irrelevant users and only calculating distances for relevant ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system calculates nearest neighbors for all users in the common embedding space, then communication targeting accuracy is improved, but computational complexity and memory overhead increase significantly
Solution Approach 1:
The patent segments the user set into two distinct groups: a first set of users who satisfy a specified condition and a second set of users who do not satisfy the condition. The system then calculates nearest neighbors only for users in the first set, excluding users in the second set from the computation. This segmentation reduces the computational burden while maintaining targeting accuracy for the relevant user population.
Solution Approach 2:
The patent extracts and removes users who do not satisfy the specified condition from the embedding space computation. By taking out the irrelevant users (second set) from the nearest neighbor calculation, the system reduces memory overhead and computational complexity while preserving the accuracy of communication targeting for the relevant users (first set).
2Adaptability or versatility
If the system includes all users in the embedding space, then the coverage of communication targeting is maximized, but memory overhead increases
Solution Approach 1:
The patent segments users into two categories based on whether they satisfy a specified condition. The system maintains the first set of users (satisfying the condition) in the embedding space for communication targeting, while excluding the second set of users (not satisfying the condition) from the embedding space. This segmentation preserves targeting coverage for relevant users while reducing overall memory requirements.
Solution Approach 2:
The patent extracts and removes users who do not satisfy the specified condition from the embedding space representation. By taking out the irrelevant users from the data structure, the system reduces memory overhead while maintaining comprehensive coverage for the relevant user population that actually receives communications.
3Measurement precision
If the system calculates distances for all users, then nearest neighbor matching accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies a preliminary filtering action by specifying a condition that pre-selects which users should be included in the nearest neighbor calculation. Users satisfying the condition are identified in advance and included in the embedding space, while users not satisfying the condition are excluded before distance calculation begins. This preliminary action reduces the number of distance calculations required while maintaining matching accuracy for the target population.
Data Source
AI summary
In some implementations, a machine learning system may receive a set of first data structures associated with a set of users, may receive a set of second data structures associated with a set of entities, and may receive event information associated with the set of users and the set of entities. The machine learning system may generate an embedding space that represents the set of first data structures, the set of second data structures, and the event information. The machine learning system may disregard a portion of the set of second data structures, using a condition, in order to generate a subset of second data structures. The machine learning system may identify at least one relevant entity, from the subset of second data structures, using the embedding space. The machine learning system may generate a communication, associated with the at least one relevant entity, and may output the communication.