Entity Linking Search Tree for Faster Large-Scale Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ML systems face inefficiencies in resource consumption and time required for matching large datasets, such as matching bank statements to invoices, due to the large search space of potential matches, which increases computational load.
Innovation Solution
A machine learning system utilizing token embeddings and a search tree to reduce the search space by identifying similar tokens and defining a revised search space, thereby reducing the number of target items to be processed, using a Trie data structure and transformer models for efficient matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large search space of target items is used for matching, then matching accuracy is improved, but resource consumption and inference time increase significantly
Solution Approach 1:
The patent segments the matching process into two distinct phases: (1) a filtering phase that quickly eliminates non-matching target items using efficient search algorithms, and (2) a refinement phase that applies comprehensive matching criteria to the reduced subset. This segmentation allows the system to maintain high matching accuracy while significantly reducing resource consumption and inference time by avoiding exhaustive comparison with all target items.
Solution Approach 2:
The patent performs preliminary filtering of the target item search space before conducting the main matching operation. By pre-processing the target items and organizing them in a searchable structure, the system eliminates obviously non-matching items in advance, thereby reducing the computational burden of the subsequent matching phase while preserving the ability to identify accurate matches.
2Use of energy by moving object
If token embeddings and search tree are used to reduce search space, then resource consumption is reduced, but matching precision may be compromised
Solution Approach 1:
The patent employs dynamic adjustment of the search space based on the query characteristics. The search tree structure allows the system to adaptively traverse only the relevant portions of the target item space, dynamically expanding or contracting the search area based on the embedding similarity scores. This dynamic approach ensures that resource consumption is minimized while maintaining matching precision by focusing computational effort on the most promising candidates.
Solution Approach 2:
The patent changes the parameter of search space size dynamically during the matching process. By using token embeddings to compute similarity scores and adjusting the threshold for inclusion in the reduced search space, the system can control the trade-off between resource consumption and matching precision. The parameter changes allow the system to maintain high precision by retaining sufficiently similar items while excluding clearly dissimilar ones, thereby reducing overall resource usage.
3Loss of time
If the search space is reduced using ML models, then inference time is reduced, but the complexity of the system increases
Solution Approach 1:
The patent segments the system into distinct functional modules: an embedding generation module that creates token embeddings, a search tree module that manages the target item structure, and a matching module that performs the actual comparison. This segmentation allows each component to be optimized independently and reused across different matching tasks, thereby reducing overall system complexity despite the advanced techniques employed.
Solution Approach 2:
The patent implements universal components that serve multiple functions. The search tree structure, for example, serves both as an organization mechanism for target items and as a filtering mechanism for reducing the search space. The token embedding approach is universally applicable to different types of items and queries, making the system highly adaptable without requiring separate specialized components for each use case, thus managing complexity effectively.
Data Source
AI summary
Methods, systems, and computer-readable storage media for a ML system that reduces a number of target items from consideration as potential matches to a query item using token embeddings and a search tree.