User Engagement Trend Detection via Amplitude and Phase Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining user engagement metrics in web services face challenges in choosing appropriate criteria that provide clear interpretations and high statistical significance, especially when subtle modifications are tested or only a small amount of user traffic is affected, leading to low metric sensitivity.
Innovation Solution
A computer-implemented method that calculates amplitude and phase metrics for user engagement by performing a discretization transform on user interactions, allowing for the determination of trends in user engagement metrics by analyzing differences between test and control groups, thereby identifying the effect of experimental treatments on user engagement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If mean values of user behavior metrics are used to evaluate performance, then the evaluation is simple and intuitive, but the metric sensitivity is low when testing subtle modifications or small user traffic changes
Solution Approach 1:
The patent transforms the user behavior metric from a simple mean value parameter to a periodicity-based parameter through discretization transform. This parameter change enables the metric to capture subtle temporal patterns in user engagement that mean values miss, thereby improving sensitivity while maintaining analytical simplicity through the periodicity framework.
2Reliability
If the experimental period is extended to improve statistical significance, then the detection power increases, but the mean value metric may not reflect the general trend of user engagement over time
Solution Approach 1:
The patent applies periodic action by using discretization transform to decompose user engagement patterns into periodic components. This allows the metric to capture recurring engagement patterns over the experimental period, enabling trend detection that is both statistically significant and informative about the direction and nature of engagement changes over time.
3Ease of manufacture
If traditional criteria are used to compare control and test groups, then the comparison method is straightforward, but the ability to detect subtle treatment effects is limited
Solution Approach 1:
The patent changes the comparison parameter from mean values to periodicity metrics. By transforming the metric to capture temporal patterns and periodicity, the comparison method maintains straightforward calculation through averaging periodicity values while dramatically improving the ability to detect subtle treatment effects that manifest as changes in engagement patterns rather than mean levels.
Data Source
AI summary
A computer-implemented method and a server with a processor are presented for determining a trend of a user engagement metric with respect to a web service. The method comprises receiving a plurality of user device requests, providing a test version of the web service to a test group and a control version of the web service to a control group, acquiring an amplitude metric and a phase metric for each one of the user devices of at least the test group, determining average group metrics, and determining the trend of the user engagement metric with respect to the web service, the determining the trend being based on analyzing of the control average amplitude metric and the test average amplitude and phase metrics.