Incrementality Model Training With Asymmetric Budget Splits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learned models for estimating the additional advancement in performance by increasing a budget for sponsored content are inaccurate due to biases in historical data, leading to unreliable budget recommendations.
Innovation Solution
A machine-learned model is trained using an asymmetric budget split process to create two groups of training data, one for high and one for low budgets, comparing performance results in each subgroup to estimate incrementality without bias, ensuring unbiased forecasting of performance-based metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If historical data is used to train the machine-learned model, then the model can be trained with available data, but the estimates become inaccurate due to biases in the historical data
Solution Approach 1:
The user base is segmented into two distinct subgroups (first subgroup and second subgroup) through random assignment. Each subgroup is exposed to different budget levels (high budget vs. low budget) for the same job postings, allowing independent measurement of performance metrics in each segment without contamination from historical biases
Solution Approach 2:
The patent applies asymmetric budget allocation where one subgroup receives high budgets and another receives low budgets for identical job postings. This intentional asymmetry creates a controlled experiment that isolates the effect of budget level from other confounding factors present in historical data, enabling unbiased incrementality estimation
2Productivity
If high budgets are applied to all job postings, then more applications may be obtained, but the cost increases and accuracy of estimating true incrementality decreases due to bias
Solution Approach 1:
The patent divides the user population into separate segments (subgroups) that are randomly assigned to different budget treatments. This segmentation allows simultaneous collection of performance data under both high and low budget conditions without mixing the effects, preserving unbiased incrementality information while maintaining overall productivity
Solution Approach 2:
Instead of applying high budgets universally, the patent applies high budgets only to a portion of users (first subgroup) while applying low budgets to another portion (second subgroup). This partial application of excessive budgeting (high budget) to only some users allows measurement of true incrementality while controlling overall costs
Data Source
AI summary
In an example embodiment, a machine-learned model is trained to forecast a performance-based metric for a piece of content based on a budget applied to the piece of content. The increase in the performance-based metric that is due to a corresponding increase in budget may be termed “incrementality.” The machine-learned model is trained in such a way that incrementality is built into the model. More particularly, in an example embodiment, an asymmetric budget split process is used to create two groups of training data, one for high budget and one for low budget. Rather than relying on historical data, the asymmetric budget split process applies a high budget to a piece of content in a first subgroup (e.g., group of users) and a low budget to that same piece of content in a second subgroup, and then the performance results in each subgroup are compared.