Search Query Auto-Completion from Unstructured Text Using NER
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information retrieval systems face challenges in providing up-to-date auto-complete and auto-correct functionalities due to the rapidly evolving nature of searchable items, limited availability of current search queries, and regulatory constraints, leading to inefficient resource usage and user iterations.
Innovation Solution
Implementing a Bidirectional Encoder Representations from Transformers (BERT)-based model and a ranked trie-tree (RTT) to enable dynamic, resource- and time-efficient auto-completion and auto-correction of search queries, utilizing named-entity recognition (NER) and a distilBERT-based model to handle spelling errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional auto-complete and auto-correct functions are implemented using conventional methods, then basic functionality is provided, but the system cannot keep up with rapidly evolving searchable items and consumes excessive computational resources
Solution Approach 1:
The patent applies preliminary action by pre-training the BERT model on large corpora of searchable items and their descriptions before deployment. This pre-training enables the model to understand entity relationships and contexts in advance, so when new searchable items are added, the system can quickly adapt without requiring intensive real-time computational resources for basic understanding tasks.
Solution Approach 2:
The patent introduces an intermediary layer between the searchable items database and the auto-complete function. The BERT model acts as this intermediary, processing and understanding the semantic relationships between items, descriptions, and entities. This intermediary enables the system to handle evolving items efficiently by leveraging the model's pre-learned knowledge rather than processing everything from scratch.
2Measurement precision
If comprehensive training data is used to improve auto-complete accuracy, then suggestion quality improves, but training time and computational overhead increase
Solution Approach 1:
The patent performs preliminary action by curating and preparing training data in advance, including collecting descriptions of searchable items, identifying entities within those descriptions, and creating labeled training examples before model training begins. This preparation work is done offline, allowing the actual model training to proceed more efficiently with ready-to-use, high-quality data.
Solution Approach 2:
The patent applies self-service by implementing automated entity recognition and data labeling processes that reduce manual intervention. The system automatically processes searchable item descriptions to identify entities and generate training data labels, thereby reducing the time and resources required for manual data preparation while maintaining high training data quality.
3Speed
If real-time processing is implemented to provide timely suggestions, then user experience improves, but computational complexity increases
Solution Approach 1:
The patent segments the auto-complete processing into distinct components: the BERT model handles semantic understanding and entity recognition, while a separate ranking system orders the suggestions. This segmentation allows each component to be optimized independently - the BERT model can be pre-trained and cached, while only the inference and ranking operations need to run in real-time, reducing overall computational complexity during user interactions.
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving a set of descriptions provided as unstructured data, each description associated with one or more entities in a set of entities that can be queried using the IR system, providing, from the set of descriptions, a first set of training data including at least a first set of entities including at least a portion of the set of entities, training a named-entity recognition (NER) model using at least a portion of the first set of training data, receiving, by the IR system, a portion of a search query, and providing a set of auto-complete suggestions based on the portion of the search query and the NER model.