Query Language Identification via Multi-Parameter Classifier
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Search engines face challenges in determining the language of search queries accurately, especially when query terms are not in the same language as the user interface, leading to suboptimal results due to language dependencies and misclassifications.
Innovation Solution
A method is developed to determine the language of a search query by generating a query language classifier based on past query records, using a training corpus to score the likelihood of query terms in different languages, and combining these scores with user history to accurately identify the query language, even with borrowed words or pair-wise combinations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the search engine uses the interface language to determine query language, then the process is simple, but the accuracy deteriorates when query terms are in a different language than the interface
Solution Approach 1:
The patent introduces an intermediary language classification system that analyzes query terms independently of the interface language. This intermediary mechanism compares query terms against multiple language corpora and uses probability scoring to determine the actual language of the query, resolving the conflict between simple interface-based determination and accurate language identification.
Solution Approach 2:
The system changes the parameter of language determination from relying solely on interface language to using a multi-parameter approach that includes term frequency analysis, probability scoring across multiple languages, and user history data. This transforms a single-parameter simple check into a multi-parameter accurate classification system.
2Reliability
If the search engine augments queries with language-dependent variants, then recall improves, but precision deteriorates when the query language is misidentified
Solution Approach 1:
The patent implements feedback mechanisms where the language classification system continuously learns from user interactions and query results. When augmentation leads to poor results, the system adjusts its language probability scores and classification parameters, creating a feedback loop that improves both language identification accuracy and augmentation effectiveness over time.
Solution Approach 2:
The language classification system is made dynamic by allowing probability scores and language preferences to change based on user behavior, query patterns, and result feedback. This dynamic adaptation enables the system to optimize both recall through appropriate augmentation and precision through accurate language identification simultaneously.
3Measurement precision
If the system considers multiple languages and user history, then language identification accuracy improves, but computational complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-processing and indexing query terms across multiple language corpora before actual query processing. Language probability scores and user history data are pre-computed and stored, allowing the classification system to make accurate determinations without performing complex real-time calculations, thus reducing operational complexity while maintaining high accuracy.
Data Source
AI summary
Methods, systems, and apparatus, including computer program products, for identifying the language of a search query. In one embodiment, the language of each term of a query is determined from the query terms and the language of the user interface a user used to enter the query. In another embodiment, an automatic interface language classifier is generated from a collection of past queries each submitted by a user. In some embodiments, a score is determined for each of multiple languages, each score indicating a likelihood that the query language is the corresponding one of the multiple languages.