Spatial-Temporal Random Segmentation for Multi-Variant Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing testing methods for multiple variants in scenarios where decision or behavior interference occurs, such as in ride-hailing platforms, face challenges with spatial and temporal interference biases, leading to inaccurate evaluation of treatment effects.
Innovation Solution
A spatial-temporal random segmentation testing method that divides testing units into grid cells and intervals, rotating feature assignments to minimize interference by determining optimal spatial and temporal granularities based on treatment effect differences, ensuring independence among testing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If random splitting of testing units is used to evaluate treatment effects, then the testing method is simple and widely applicable, but spatial interference bias occurs when testing units are interdependent
Solution Approach 1:
The patent segments the testing units into multiple spatial clusters or geographic regions, and further divides time into multiple time periods. By creating a matrix of spatial clusters × time periods, the system generates multiple testing groups that rotate through different feature variants. This segmentation approach maintains independence between testing units within each group while allowing comprehensive evaluation across all segments, thereby eliminating spatial interference bias without significantly increasing complexity.
2Adaptability or versatility
If testing units are divided into groups to test multiple variants, then treatment effect evaluation becomes more comprehensive, but interference bias increases when units are spatially or temporally dependent
Solution Approach 1:
The patent implements periodic action by rotating the assignment of feature variants across different time periods. Each spatial cluster tests different variants in different time periods, and the system periodically cycles through all variant assignments. This periodic rotation ensures that each spatial cluster eventually tests all variants while preventing simultaneous interference between comparable units, thereby eliminating both spatial and temporal interference bias while maintaining comprehensive variant evaluation.
3Measurement precision
If spatial and temporal granularities are optimized to reduce interference bias, then measurement precision improves, but system complexity increases
Solution Approach 1:
The patent implements self-service through automated algorithms that dynamically determine optimal spatial and temporal granularities. The system automatically calculates the appropriate number of spatial clusters and time periods based on the total number of testing units and desired confidence levels, then self-generates the random assignment schedules. This automation eliminates the need for manual configuration and reduces operational complexity while maintaining high measurement precision through optimized granularity selection.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for testing a plurality of variants among a plurality of users. One of the methods includes: determining a spatial granularity to divide an area into a plurality of grid cells; randomly splitting the plurality of grid cells into a plurality of testing groups, wherein a quantity of the plurality of testing groups is determined based on a quantity of the multiple versions of the feature to be tested; determining a temporal granularity to divide a testing period into a plurality of testing intervals; during each of the plurality of testing intervals, respectively assigning the multiple versions of the feature to the plurality of testing groups; and obtaining a treatment effect for each of the multiple versions of the feature and determining an optimal version of the feature.