Query Result Sampling by Search Frequency for Unbiased Engine Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing query engines in large-scale databases face challenges in accurately evaluating performance due to biased sampling of historical query records, particularly when common search phrases dominate, leading to skewed results and subjective variations among users.
Innovation Solution
The system stratifies historical query records based on search frequencies and samples a representative set from these bins to reduce bias, enabling unbiased evaluation and refinement of the item query engine.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all historical query records are used to evaluate query engine performance, then the evaluation comprehensiveness is improved, but the processing time and computational resources increase significantly
Solution Approach 1:
The patent segments the historical query records into different strata based on search frequency categories (high, medium, low). This segmentation allows the system to evaluate a representative sample from each stratum rather than processing all records, thereby reducing processing time while maintaining evaluation comprehensiveness through balanced coverage of different query types.
2Productivity
If random sampling is used to select historical query records, then the processing efficiency is improved, but the evaluation accuracy deteriorates due to bias toward common search phrases
Solution Approach 1:
The patent changes the sampling parameter from uniform random sampling to stratified sampling based on search frequency parameters. By categorizing queries into different frequency strata and sampling proportionally from each stratum, the system eliminates the bias toward common search phrases while maintaining processing efficiency through controlled sample sizes.
3Measurement precision
If stratified sampling based on search frequency is implemented, then the evaluation accuracy is improved by reducing bias, but the system complexity increases
Solution Approach 1:
The patent implements segmentation of query records into frequency-based strata with predefined sampling proportions. This structured approach improves evaluation accuracy by ensuring representative coverage of rare and common queries while managing system complexity through automated stratification logic and proportional sampling rules.
4Speed
If common search phrases dominate the sample, then the processing speed is improved, but the result relevance deteriorates due to skewed evaluation
Solution Approach 1:
The patent changes the sampling distribution parameter from uniform to stratified based on search frequency. This ensures that rare search phrases are adequately represented in the sample while maintaining reasonable processing speed through controlled sample sizes from each stratum, thereby improving result relevance without excessive processing time.
Data Source
AI summary
An online system may generate numerous search records in response to searches requested by users. The online system may use a specific way to sample the historical search records to reduce biases in sampling. For example, the online system retrieves historical query records associated with an item query engine. The set of historical query records includes a plurality of search phrases. A historical query record is associated with a search phrase and a list of items returned by the item query engine. The online system determines the search frequencies for the search phrases. The online system stratifies the historical query records into a plurality of bins according to the search frequencies of the search phrases. The online system samples the historical query records from the plurality of bins to collect a representative set of historical query records and outputs the representative set of historical query records for rating.