Online Ad Experimentation System Using Prediction Model Confidence

Solution Overview

Problem

The challenge lies in efficiently conducting experiments with multiple content variations, as the number of test subjects and variations required grows exponentially, making the experimentation process time-consuming and costly, potentially rendering it unaffordable for achieving desired objectives such as maximizing product sales.

Innovation Solution

A system comprising a communication management system and a content system that optimizes experimentation by delivering varying content treatments to control groups, using a prediction model to determine confidences and apply weighting factors, thereby reducing the time to identify optimal content combinations and minimizing the delivery of sub-optimal content.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the number of alternate ideas and test subjects is increased to obtain accurate test results, then the accuracy of experimentation is improved, but the time and cost required for administering the experiment increases significantly

Engineering Contradiction:

Improveaccuracy of test resultsVSAvoidtime required for experimentation

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary actions by using a prediction model to estimate treatment outcomes before conducting the full experiment. This allows the system to prioritize which treatment-subject combinations are most likely to be informative, thereby reducing the number of combinations that need to be tested while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses its own accumulated data and prediction model to guide the experimentation process. By continuously learning from observed outcomes and using this information to select future treatment assignments, the system serves itself in optimizing the experiment design, reducing both time and resource requirements.

Inventive Principle:

Principle #25Self-service

2Measurement precision

If the number of alternate ideas and test subjects is increased to obtain accurate test results, then the accuracy of experimentation is improved, but the cost of administering the experiment increases significantly

Engineering Contradiction:

Improveaccuracy of test resultsVSAvoidcost of experimentation

Core Design Contradiction:

Measurement precisionVSLoss of energy

Solution Approach 1:

The system performs preliminary actions by using a prediction model to estimate treatment outcomes before conducting the full experiment. This allows the system to prioritize which treatment-subject combinations are most likely to be informative, thereby reducing the number of combinations that need to be tested while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses its own accumulated data and prediction model to guide the experimentation process. By continuously learning from observed outcomes and using this information to select future treatment assignments, the system serves itself in optimizing the experiment design, reducing both time and resource requirements.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If the number of treatment variations is increased to test more content combinations, then the comprehensiveness of experimentation is improved, but the complexity of administering the experiment increases

Engineering Contradiction:

Improvecomprehensiveness of content testingVSAvoidcomplexity of experiment administration

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by using a prediction model to estimate treatment outcomes before conducting the full experiment. This allows the system to prioritize which treatment-subject combinations are most likely to be informative, thereby reducing the number of combinations that need to be tested while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by dynamically adjusting the assignment probabilities of different treatments based on confidence scores generated by the prediction model. This allows the system to adaptively explore the treatment space, focusing resources on the most promising variations while maintaining comprehensiveness.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS9002729B2System and method for determining sets of online advertisement treatments using confidences

Publication Date: 2015.04.07 ACCENTURE GLOBAL SERVICES LTD

AI summary

A system for experimentation includes an experiment engine which can define an experiment relating to various treatments for a set of content elements. The experiment engine conducts the experiment by delivering treatments to control groups over the network. The selection of the treatments to deliver is optimized to provide treatments having a greater likelihood of satisfying an objective of the experiment.