Abbreviated Term Expansion Using Search Probability Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional large language models (LLMs) struggle with accurately predicting expanded terms from abbreviated search queries due to a lack of context, leading to inefficient search results, particularly in systems like SKU product search systems where users expect precise product names.
Innovation Solution
A neural network is trained using probabilistic tuples derived from user telemetry data to associate abbreviated terms with their most likely expanded terms, dynamically changing input data during training to improve prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional LLMs are used to predict expanded terms from abbreviated queries, then the system can process search queries, but the prediction accuracy deteriorates due to lack of context and domain-specific knowledge
Solution Approach 1:
The system performs preliminary action by pre-computing and storing expanded term probabilities for abbreviated terms based on historical search data and telemetry. This pre-processing step enriches the abbreviated terms with contextual information before the actual search query is processed, allowing the LLM to make more accurate predictions without requiring additional context during inference.
Solution Approach 2:
The patent introduces an intermediary mechanism that enriches abbreviated terms with expanded term probabilities before they are processed by the LLM. This intermediary layer bridges the gap between the abbreviated query and the LLM's prediction capability, enabling accurate expanded term identification without requiring the LLM to infer context from scratch.
2Productivity
If simpler NLP techniques are used instead of LLMs, then the system becomes more efficient and requires less context, but the search result accuracy deteriorates for domain-specific queries
Solution Approach 1:
The patent merges the strengths of simpler NLP techniques and LLMs by combining abbreviated term expansion with LLM-based semantic search. The system uses enriched abbreviated terms (with expanded term probabilities) as input to the LLM, allowing the LLM to leverage its contextual understanding capabilities while maintaining efficiency. This hybrid approach combines the speed of simpler NLP methods with the accuracy of LLMs.
3Adaptability or versatility
If LLMs trained on public information are used, then the system can process general queries, but the performance deteriorates for domain-specific abbreviated terms like SKU products
Solution Approach 1:
The patent applies local quality by customizing the LLM's behavior for specific domain contexts. The system enriches abbreviated terms with domain-specific expanded term probabilities (e.g., product names for SKU queries) before processing. This allows the LLM to maintain its general query processing capability while adapting to domain-specific terminology and expectations through the enriched input data.
Solution Approach 2:
The system changes the parameters of the LLM input by transforming plain abbreviated terms into enriched terms that include expanded term probabilities. This parameter transformation allows the LLM to operate in a domain-specific context without retraining, by modifying the input representation to reflect domain knowledge and statistical relationships between abbreviated and expanded terms.
Data Source
AI summary
Systems and methods for training a neural network architecture to infer an expanded term associated with an abbreviated term includes determining, based on prior search data, a probability associated with each expanded term of a plurality of expanded terms related to an abbreviated term, during a training epoch associated with training a neural network architecture: determining, based on the probability associated with each expanded term of the plurality of expanded terms related to the abbreviated term, input data associated with the training epoch; and performing the training epoch on the neural network architecture using the input data. Methods further include applying the neural network architecture to a search comprising the abbreviated term, wherein the neural network architecture infers an expanded term for the abbreviated term based on the search.