Experimental Parameter Testing Using Quantile Indicators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating the impact of new functions on traffic indicators in applications are inaccurate due to the use of average values from different databases, which fail to accurately reflect the distribution of traffic indicators stored in quantiles.
Innovation Solution
A method and apparatus for experimental parameter testing that determine a traffic indicator, obtain traffic indicator values from multiple databases, calculate a quantile indicator value, and assess test results based on ground-truth indicator values from experimental and control groups, using permeability indicators to evaluate the optimization or degradation of experimental parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If average values from different databases are used to evaluate traffic indicators, then the evaluation process is simple, but the accuracy of experimental evaluation is low
Solution Approach 1:
The patent changes the evaluation parameter from average values to quantile values (e.g., 50th quantile). This parameter transformation allows the system to accurately reflect the distribution characteristics of traffic indicators stored in different database formats (both average values and quantiles) while maintaining computational feasibility through the construction of a quantile evaluation model.
2Loss of information
If quantile values are used to represent traffic indicators, then the distribution information is preserved, but the accuracy of average-based evaluation deteriorates
Solution Approach 1:
The patent segments the evaluation process into distinct components: extracting quantile values from different databases, constructing a quantile evaluation model, and comparing quantile indicators. This segmentation allows the system to handle mixed database formats (average values and quantiles) separately and appropriately, preserving distribution information while achieving accurate evaluation.
Solution Approach 2:
The patent introduces a quantile evaluation model as an intermediary between the raw traffic indicator data and the final evaluation result. This intermediary model standardizes the comparison by transforming various database formats into a unified quantile-based evaluation framework, enabling accurate comparison while preserving distribution characteristics.
3Ease of operation
If ground-truth indicator values from experimental and control groups are compared using average values, then the comparison is straightforward, but the test result accuracy is low
Solution Approach 1:
The patent creates a universal quantile evaluation model that can handle multiple database formats (both average value databases and quantile databases) through a unified comparison framework. This multi-functional approach maintains operational simplicity by providing a single evaluation process while improving accuracy through quantile-based comparison of ground-truth indicator values from experimental and control groups.
Data Source
AI summary
The disclosure provides a solution for experimental parameter testing. A method includes: determining a traffic indicator of a target object, and obtaining a plurality of traffic indicator values of the traffic indicator from a plurality of databases; determining, based on the plurality of traffic indicator values, a quantile indicator value corresponding to the traffic indicator; obtaining a plurality of first ground-truth indicator values corresponding to an experimental group and a plurality of second ground-truth indicator values corresponding to a control group of a traffic indicator of the plurality of databases; and determining a test result of the experimental parameter based on the plurality of first ground-truth indicator values, the plurality of second ground-truth indicator values and the quantile indicator value, the test result being test passed or test failed.