Query Similarity Determination via Distribution Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information searching systems face challenges in determining the similarity of queries, particularly across languages, due to the ambiguity of query terms and the limitations of conventional translation methods such as bilingual dictionaries and parallel corpora, which often result in noisy translations and high costs.
Innovation Solution
A method that involves logging and comparing query distributions over time to identify similar queries by determining the volume of query issuances per unit time, allowing for the designation of queries as similar based on their distribution patterns, regardless of language differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If bilingual dictionaries are used to translate queries, then query translation is achieved, but translation quality deteriorates due to noisy translations from multiple dictionary senses
Solution Approach 1:
Instead of copying literal translations from dictionaries, the system copies the query distribution pattern (temporal and volumetric characteristics) from similar queries in the target language. This allows the system to infer translation equivalents by observing when and how often similar queries are issued, rather than relying on noisy dictionary translations.
Solution Approach 2:
The system uses feedback from query logs to continuously refine query similarity determination. By analyzing actual query patterns and user behavior data, the system improves its ability to identify similar queries across languages without relying on pre-computed dictionary translations.
2Measurement precision
If parallel corpora are used to translate queries, then translation accuracy improves, but system complexity and cost increase due to the expense of building and maintaining such corpora
Solution Approach 1:
The system performs self-service by utilizing its own query logs and operational data to determine query similarity. Instead of requiring external parallel corpora or expensive translation resources, the system leverages its native query distribution patterns to identify similar queries across languages.
Solution Approach 2:
The system replaces expensive, complex parallel corpora with inexpensive, readily available query log data. The query distribution patterns are derived from existing operational logs, eliminating the need for costly resource investment in building and maintaining parallel language corpora.
3Measurement precision
If query distribution comparison is used to identify similar queries, then translation accuracy improves without dictionaries, but data processing complexity increases due to logging and analysis requirements
Solution Approach 1:
The system performs preliminary action by continuously logging and pre-processing query data in the background. Query distributions are computed and stored in advance, so when similarity determination is needed, the system can quickly compare pre-computed distributions without performing complex real-time analysis.
Solution Approach 2:
The system segments the query analysis process into distinct components: logging query issuances, computing query distributions (temporal and volumetric), and comparing distributions to identify similarities. This segmentation allows each component to be optimized independently and reduces overall processing complexity.
Data Source
AI summary
A system determines query similarity. The system determines a volume per unit time of an issued first query over a time period and determines a volume per unit time of issued other queries over the time period. The system compares the volume per unit time of each of the issued other queries to the volume per unit time of the issued first query. The system identifies ones of the issued other queries as similar to the first query based on the comparison.