Named Entity Recognition for Multi-Token Search Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search management systems fail to accurately identify and process multi-token phrases in search queries, leading to irrelevant search results by treating individual components of a phrase as separate search terms rather than a single unit.
Innovation Solution
Implementing a machine-learning based named entity recognition model that identifies and processes multi-token phrases as single entities, excluding individual terms and returning results based on the entire phrase, thereby enhancing the accuracy of search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If search systems process individual terms separately, then search coverage is increased, but search accuracy deteriorates by returning irrelevant results
Solution Approach 1:
The system segments the search query into individual tokens and identifies which sequences should be treated as multi-token phrases. By dividing the query processing into token-level analysis and phrase-level matching, the system can simultaneously maintain search coverage through individual term matching and search accuracy through phrase integrity preservation.
Solution Approach 2:
The system introduces an intermediary named entity recognition model that acts as a mediator between the raw search query and the search indexing system. This model identifies multi-token phrases and provides guidance to the search system on how to process them, preventing the breakdown of phrases into individual terms during search execution.
2Device complexity
If search systems treat multi-token phrases as individual terms, then processing complexity is reduced, but result relevance deteriorates
Solution Approach 1:
The system performs preliminary action by pre-identifying multi-token phrases in the search query using a named entity recognition model before executing the search. This advance identification allows the system to maintain phrase integrity throughout the search process without adding complexity during the actual search execution, as the phrase boundaries are already established.
3Device complexity
If search systems use traditional term-based processing, then system simplicity is maintained, but search result quality deteriorates
Solution Approach 1:
The named entity recognition model serves as an intermediary layer between the user's search query and the traditional search processing system. This intermediary identifies and marks multi-token phrases without requiring fundamental changes to the underlying search infrastructure, thus maintaining system simplicity while improving search result quality through better phrase handling.
Data Source
AI summary
A system and method to receive a search query including a set of search terms associated with a merchant system. A machine-learning model is executed to identify a first subset of one or more multi-term phrases associated with one or more named entity types. A set of tokens corresponding to the search query is generated, wherein the set of tokens comprises a token associated with each of the first subset of one or more multi-term phrases. A comparison of the set of tokens to a document index associated with the merchant system is executed to identify one or more matching documents. Based on the comparison, a set of search results comprising the one or more matching documents is generated.