Search Query Phrase Entropy Calculation for Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engines struggle to accurately determine the importance of words in short, informal queries, leading to inadequate distinction between relevant and irrelevant search results due to the limitations of the TF/IDF method in calculating word information entropy.
Innovation Solution
A process that categorizes queries, segments them into phrases, calculates occurrence probabilities, and determines word information entropy for each phrase, allowing for the storage and updating of correspondence relationships between phrases and their entropies, which are then used to refine search results by focusing on more important phrases when close matches are not found.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the TF/IDF method is used to calculate word information entropy, then the calculation process is simple, but the ability to distinguish word importance in short queries is insufficient
Solution Approach 1:
The query is segmented into multiple phrases at different levels (first-level phrases and second-level phrases). This segmentation allows the system to calculate word information entropy for different phrase levels separately, enabling more precise distinction of word importance within the context of short queries while maintaining the simplicity of the TF/IDF calculation method for each segment.
2Productivity
If the query is processed as a whole, then the processing is efficient, but the distinction between relevant and irrelevant search results is inadequate
Solution Approach 1:
The query is divided into multiple phrases (first-level phrases and second-level phrases) rather than processing as a whole. This segmentation enables the system to calculate word information entropy for different phrase levels, identify key phrases with higher importance, and perform more targeted searching, thereby improving search result relevance while maintaining processing efficiency through hierarchical organization.
Solution Approach 2:
Different phrases within the query are assigned different levels of importance based on their word information entropy. The system identifies key phrases (those with higher importance) and focuses the search on these specific local parts of the query, rather than treating all words equally. This local quality approach improves search result relevance by concentrating on the most informative parts of the query.
Data Source
AI summary
Determining and using word information entropies includes: determining one or more categories that correspond to a plurality of queries; sorting the plurality of queries into one or more groups based at least in part on the determined categories of the plurality of queries; segmenting queries that correspond to each of the one or more groups into a first plurality of phrases, wherein each phrase includes one or more words; determining occurrence probabilities for the plurality of phrases; and determining word information entropies for the plurality of phrases based at least in part on the determined occurrence probabilities.