Surrogate Metric Selection for Web Design Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing website and computer application design changes are inefficient and inaccurate, as they often require large amounts of data and struggle to determine the effectiveness of design changes based on user engagement metrics.
Innovation Solution
The use of surrogate metrics that are identified through past experiment data to indicate user experience and engagement, allowing for more efficient and accurate testing by selecting the most sensitive and correlated metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional user testing methods are used to determine design effectiveness, then accurate user engagement data can be obtained, but the testing process becomes inefficient and requires large amounts of data
Solution Approach 1:
The patent introduces surrogate metrics as intermediary indicators that indirectly reflect user engagement. Instead of directly measuring complex user engagement behaviors, the system uses proxy metrics (such as interaction frequency, time spent, or completion rates) that are easier to collect and process, thereby improving testing efficiency while maintaining measurement accuracy
Solution Approach 2:
The patent performs preliminary analysis of historical testing data to identify and select optimal surrogate metrics before conducting new experiments. By pre-determining which surrogate metrics best correlate with user engagement based on past results, the system avoids trial-and-error during actual testing, significantly improving productivity
2Reliability
If traditional testing methods are used, then comprehensive user response data can be collected, but the amount of testing data required becomes excessively large
Solution Approach 1:
Surrogate metrics serve as efficient intermediaries that capture user engagement patterns with less data than direct measurement methods. These proxy indicators can reliably indicate user engagement trends using smaller sample sizes, thereby maintaining result reliability while reducing the quantity of testing data needed
Solution Approach 2:
The patent transforms the measurement approach by changing from direct user engagement parameters to surrogate metric parameters. This parameter transformation allows the system to achieve reliable testing results with reduced data quantities by leveraging the statistical properties and correlations of the surrogate metrics
3Measurement precision
If multiple surrogate metrics are evaluated, then better indication of user experience can be achieved, but the complexity of metric selection and analysis increases
Solution Approach 1:
The patent performs preliminary evaluation and ranking of surrogate metrics based on historical data analysis before actual testing. By pre-identifying the most relevant and reliable surrogate metrics through correlation analysis with user engagement, the system simplifies the metric selection process while maintaining high measurement precision
Solution Approach 2:
The patent segments the metric evaluation process into distinct stages: surrogate metric identification, correlation analysis with user engagement, and selection of optimal metrics. This segmentation organizes the complex task into manageable steps, reducing analytical complexity while improving user experience indication accuracy
Data Source
AI summary
The disclosure includes methods and an apparatus that includes processing circuitry that selects at least one candidate surrogate metric from a plurality of surrogate metrics based on first testing data of a target metric and the plurality of surrogate metrics from a first database in memory. The first testing data have been generated from previously controlled testing of a control variant and a treatment variant of a feature of a webpage or a computer application. The processing circuitry determines current testing results associated with the plurality of surrogate metrics and determines an output of the current controlled testing based on one or more of the current testing results associated with the at least one candidate surrogate metric. If the output indicates the treatment variant replacing the control variant of the feature of the webpage or the computer application, the control variant is replaced with the treatment variant of the feature.