Search Term Clustering via Semantic Syntactic Distance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in managing the vast number of unique search terms used by users to express similar intents, leading to increased complexity in predicting and suggesting relevant search terms, as these terms often follow a fat-tail distribution with high skewness, making it difficult for search engines to provide an intuitive consumer experience.
Innovation Solution
The solution involves clustering search terms using both syntactic and semantic distances, where syntactic techniques analyze lexical aspects and semantic techniques consider user activity, allowing for the computation of similarity metrics and subsequent clustering to group search terms based on their intent, thereby reducing the number of unique terms and improving search engine efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If search engines process all unique search terms individually, then they can capture all user intents, but the complexity of predicting and suggesting relevant search terms increases significantly
Solution Approach 1:
The patent merges similar search terms by computing semantic similarity between them and grouping terms with similarity above a threshold into clusters. Each cluster is represented by a representative term, reducing the total number of terms to process while preserving user intent coverage. This directly addresses the contradiction by combining multiple unique terms into fewer representative groups.
Solution Approach 2:
The patent creates a universal search term representation system where clustered search terms serve multiple functions: they represent multiple original terms, enable predictive suggestions, and maintain intent coverage. The representative terms of clusters can be used universally across different search contexts, reducing complexity while preserving functionality.
2Quantity of substance
If search engines use semantic similarity computation to cluster search terms, then the number of unique terms is reduced, but the computational resources required for similarity calculation increase
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing semantic similarities between search terms, and pre-clustering terms before the actual search operation. This preliminary clustering reduces the number of terms that need to be processed during real-time search operations, thereby reducing computational resources required at query time.
Solution Approach 2:
The patent replaces direct semantic similarity computation during search operations with pre-computed cluster memberships and representative terms. Instead of computing semantic similarity on-the-fly for each query, the system substitutes this with faster lookups of pre-established cluster assignments, reducing real-time computational burden.
Data Source
AI summary
When conducting the same or similar search, different users can use different search terms and phrases, resulting in an increase in the quantity of unique search terms and phrases. The intent of the various search terms and phrases is determined based on clustering of the terms and phrases of the various users. User search terms bare clustered using semantic and syntactic distances. Thus, the search engine receives a search query from a user and computes a similarity between and among user search terms. The computation uses syntactic techniques to analyze lexical aspects of linguistic terms, and semantic techniques to consider activity of the user in the particular field of interest. A similarity metric is used to determine the similarity between two search terms by computing their syntactic and semantic distances. A clustering technique is then used to cluster search terms based on their pair-wise distance.