Experiment Campaign System for Isolating Parameter Effects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Advertisers face challenges in isolating the impact of changes to campaign parameters on advertisement performance due to external factors, making it difficult to determine whether performance changes are due to parameter adjustments or other influences.
Innovation Solution
The implementation of an experiment campaign system that allows for the creation of an experiment campaign with adjusted parameters relative to a control campaign, enabling direct comparison of performance measures between the two over a shared time period, thereby isolating the effect of parameter changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If advertisers adjust campaign parameters to improve advertisement performance, then performance may be improved, but it becomes difficult to determine whether changes are due to parameter adjustments or external factors
Solution Approach 1:
The system segments the campaign into control and experiment portions, where the control portion maintains original parameters and the experiment portion applies adjusted parameters. This segmentation allows independent measurement of each portion's performance, enabling accurate attribution of performance changes to parameter adjustments rather than external factors.
Solution Approach 2:
The system implements feedback by continuously monitoring and comparing performance measures between the control and experiment portions. This feedback mechanism provides real-time information about the impact of parameter changes, allowing advertisers to determine causality through systematic comparison rather than relying on overall campaign performance alone.
2Loss of information
If full campaign data is stored for analysis, then complete performance information is available, but data storage requirements increase
Solution Approach 1:
The system extracts only the essential data needed for comparison - specifically performance measures from the experiment portion and corresponding control portion data. By taking out only the relevant differential information rather than storing complete campaign data, the system maintains analytical completeness while significantly reducing storage requirements.
Solution Approach 2:
The system creates a simplified copy of the control portion data that corresponds only to the experiment portion's time period and parameters. This selective copying provides sufficient information for causal analysis without duplicating entire campaign datasets, thereby reducing storage volume while preserving necessary performance information.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for creating experiment content distribution campaigns (“experiment campaign”) and allocating content in accordance with the experiment campaign. In one aspect, a control campaign or an experiment campaign are selected as an available campaign for providing content in response to receiving a content request. The selection can be a probability constrained selection where the probability with which the experiment campaign is selected is specified as a portion of a total number of requests received during an experiment period for which the experiment campaign is available to provide content. Content is provided in response to the request, where the provided content being content corresponding to the selected campaign.