Client-Side Token Assignment for Consistent A/B Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
A/B testing across multiple web servers is challenging due to issues with mirroring visitor databases and timely propagation of test version information, leading to inconsistent assignment of test versions to website visitors, especially when load-balancers are involved, resulting in biased test results.
Innovation Solution
The implementation of a test management system that uses tokens stored on client devices to map test versions of web pages, ensuring consistent and randomized assignment by generating a token for each visitor and distributing the mapping across all web servers, eliminating the need for persistent storage and timely propagation of visitor information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If persistent storage mechanisms are used to maintain visitor databases across multiple web servers, then consistent test version assignment can be achieved, but system complexity and storage requirements increase significantly
Solution Approach 1:
The patent extracts the test version assignment logic from centralized persistent storage and places it in client-side tokens. Each visitor receives a token containing their assigned test version identifier, eliminating the need for servers to query databases on each request. This extraction reduces server-side complexity while maintaining assignment consistency.
Solution Approach 2:
Instead of maintaining a single source of truth in persistent storage that must be replicated across servers, the patent creates copies of assignment information in client-side tokens. Each token is a self-contained copy of the visitor's test version assignment, eliminating replication synchronization issues.
2Reliability
If visitor information is propagated across all web servers in real-time, then consistent test version assignment is maintained, but network traffic and latency increase
Solution Approach 1:
The patent performs test version assignment in advance and stores the result in the visitor's token before they encounter any latency issues. When a visitor returns, their token already contains the pre-determined test version assignment, eliminating the need for real-time propagation during their visit.
Solution Approach 2:
Visitors carry their own test version assignment information in tokens, making the system self-sufficient without requiring continuous server-to-server communication. Each server can independently process visitors using local token validation, eliminating network dependency for assignment consistency.
3Ease of manufacture
If IP address-based hashing is used to assign test versions, then simple implementation is achieved, but geographic bias in test results occurs
Solution Approach 1:
The patent changes the parameter used for assignment from IP address (geographic correlate) to random token identifiers. This parameter change eliminates the geographic bias inherent in IP-based systems while maintaining implementation simplicity through straightforward token generation and comparison logic.
Data Source
AI summary
The present disclosure is directed toward systems and methods for providing consistent test versions of an A/B tested web page in an A/B test. For example, one or more embodiments described herein generate an A/B test mapping that associates ranges of identifiers with test versions of an A/B tested web page. Additionally, embodiments described herein generate a web page visitor token based on a random number that corresponds to a single identifier within the mapping. Each time the web page visitor requests the A/B tested web page, systems and methods described herein utilize the visitor token to consistently provide the same test version of the A/B tested web page to the web page visitor.