Query Rewriting Label Acquisition via Click Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Acquiring training data for machine learning classifiers is time-consuming and costly, as it often requires human judges to manually inspect examples for classification tasks, such as query rewriting in search engines.
Innovation Solution
A system and method for automatically identifying and labeling word pairs by analyzing query data, using indicators from click analytics, behavioral targeting, geolocation, and logfile analysis to determine the relevance of alternate words, and communicating these labels to classifiers for efficient query rewriting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human judges manually inspect examples to acquire training data, then labeling accuracy is improved, but time consumption and cost increase
Solution Approach 1:
The system uses users' natural search behavior and click patterns to automatically generate labels for word pairs. Instead of requiring external human judges, the system self-services by extracting labeling information from existing user interactions with search results, including click-through data and behavioral patterns.
Solution Approach 2:
The system implements a feedback loop where user click behavior on search results provides continuous labeling signals. When users click on certain results for query variations, this feedback automatically reinforces or corrects classifier predictions, enabling ongoing improvement without additional manual labeling effort.
2Measurement precision
If human judges manually inspect examples to acquire training data, then labeling accuracy is improved, but cost increases
Solution Approach 1:
The system converts existing user interaction data into training labels at minimal cost. By leveraging free click analytics and behavioral data already collected during normal search operations, the system eliminates the need to pay human judges while maintaining labeling quality through objective user behavior signals.
Solution Approach 2:
Instead of creating new manual labels, the system copies and repurposes existing user click behavior data as training labels. The same user interactions that drive search result ranking also provide the labeling signals needed for query rewriting classification.
3Productivity
If automatic labeling methods are used, then productivity is improved, but measurement precision may worsen
Solution Approach 1:
The system uses real-time feedback from user click behavior to continuously refine and correct automatic labels. When the classifier makes predictions, subsequent user interactions provide feedback that automatically adjusts and improves labeling accuracy without reducing productivity.
Solution Approach 2:
The system makes user click data serve multiple functions simultaneously: it powers the search result ranking algorithm and also provides training labels for the query rewriting classifier. This multi-functional use of the same data source enables both high productivity and maintained accuracy.
Data Source
AI summary
Systems, methods, and computer storage media having computer-executable instructions embodied thereon for rewriting queries and labeling word pairs. Queries are received and alternate words are identified for word pairs (i.e., query words and alternate words). Word pair links are presented to users and indicators are received based on actions taken by the users. Labels are assigned to the word pairs based on the indicators and communicated to a classifier.