Contextual Bandit Delay Buffer for Email Campaign Policy Stability

The system pre-generates multiple creative variants before the campaign starts, so that when real-time selection is needed, there are already prepared options to choose from. This preliminary preparation allows the contextual bandit to work with a fixed set of options rather than generating new creatives on-the-fly, stabilizing the policy while maintaining adaptability through feedback-driven selection among pre-generated variants.

An intermediary layer is introduced between the contextual bandit and the creative generation process. This intermediary manages the feedback loop and selection process, allowing the bandit to learn from delayed or inconsistent feedback without directly controlling real-time creative generation. The intermediary buffers the instability by decoupling the learning process from the immediate selection pressure.

The system assigns preliminary feedback values to creatives based on pre-campaign data and historical performance before the campaign launches. This preliminary valuation provides a baseline that prevents early positive feedback from completely skewing the policy, as the system already has an established expectation of performance. The preliminary values act as a buffer that maintains measurement precision while allowing quick adaptation to new feedback.

The system dynamically adjusts the learning rate and feedback weighting parameters over time during the campaign. Early in the campaign, the system may weight preliminary values more heavily to prevent skewing, while later it increases the weight of actual campaign feedback. This parameter adjustment allows the system to balance quick learning with accurate feedback valuation at different stages of the campaign lifecycle.