Multi-stage Entity Resolution System Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural language processing systems face delays and payload costs due to the time-consuming process of searching entity databases and transmitting large search results, which hinders the system's responsiveness to user inputs.
Innovation Solution
Implementing a multi-stage entity resolution processing system that performs a preliminary search to determine the existence of entities in the database, sending only preliminary search results to downstream components, while performing a complete search in parallel, reducing data transmission and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a complete entity database search is performed to ensure comprehensive entity resolution, then entity resolution accuracy is improved, but system latency increases and data transmission payload increases
Solution Approach 1:
The patent segments the entity database search into two distinct stages: a preliminary search stage that performs quick filtering to identify potential entity matches, and a complete search stage that performs comprehensive verification. This segmentation allows the system to achieve accurate entity resolution while reducing overall search time by handling different aspects of the search process separately and in parallel where possible.
Solution Approach 2:
The patent implements a preliminary search stage that performs initial entity database queries before the complete search stage. This preliminary action identifies potential matches and filters the search space, allowing the subsequent complete search to focus only on relevant candidates. This reduces the overall time required while maintaining accuracy, as the preliminary search quickly eliminates non-matching entities.
2Reliability
If complete entity database search results are transmitted to downstream components, then entity resolution completeness is improved, but data transmission payload and processing overhead increase
Solution Approach 1:
The patent extracts only the essential preliminary search results and transmits them to downstream components, while retaining the complete search results in the entity resolution system. This extraction approach allows downstream components to perform their functions with sufficient information without receiving the full payload of complete search results, thereby reducing data transmission overhead while maintaining entity resolution completeness.
Solution Approach 2:
The patent introduces an intermediary mechanism where the entity resolution system acts as a mediator between the complete search process and downstream components. Instead of directly transmitting all complete search results to downstream components, the system selectively transmits only the necessary preliminary results, acting as an intermediary that filters and manages data flow to optimize both completeness and payload efficiency.
3Measurement precision
If a single-stage complete entity search is performed, then entity resolution accuracy is improved, but system productivity decreases due to sequential processing
Solution Approach 1:
The patent implements periodic action by structuring the entity resolution process as distinct, time-bound stages: preliminary search, complete search, and result transmission. Each stage has a specific time allocation and can be executed in parallel with other system operations. This periodic structure improves productivity by making the system more predictable and responsive, while still achieving accurate entity resolution through the comprehensive complete search stage.
Solution Approach 2:
The patent maintains continuity of useful action by ensuring that while the preliminary search is being performed, other system operations can proceed in parallel. The complete search stage then continues the useful action of entity resolution without interruption, building upon the preliminary results. This continuous approach maximizes system productivity by minimizing idle time and ensuring that each stage seamlessly transitions to the next, maintaining accurate entity resolution throughout.
Data Source
AI summary
Techniques for performing multi-stage entity resolution (ER) processing are described. A system may determine a portion of a user input corresponding to an entity name, and may request an entity provider component to perform a search to determine one or more entities corresponding to the entity name. The preliminary search results may be sent to a skill selection component for processing, while the entity provider component performs a complete search to determine entities corresponding to the entity name. A selected skill component may request the complete search results to perform its processing, including determining an output responsive to the user input.