Targeting Model Evaluation Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional advertising systems lack comprehensive benchmarks and tools for evaluating advertisement targeting models, which hampers the effectiveness of ad delivery and user engagement.
Innovation Solution
An online system provides a framework for evaluating targeting models by predicting user groups based on characteristics, conducting surveys and ad preference tools to assess model accuracy, and performing A/B testing to compare model performance using metrics like ad score and click-through rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional advertising systems use targeting models without comprehensive benchmarks, then the system can operate with simpler evaluation processes, but the effectiveness of ad delivery and user engagement deteriorates
Solution Approach 1:
The patent introduces an intermediary evaluation framework that acts as a mediator between the targeting model and the advertising system. This framework includes benchmark tools, survey mechanisms, and A/B testing infrastructure that objectively assess model performance without requiring fundamental changes to the core advertising delivery system, thus improving reliability while maintaining manageable complexity
Solution Approach 2:
The patent implements feedback mechanisms through surveys administered to users and performance tracking via A/B testing. These feedback loops provide quantitative data on model accuracy, precision, and recall, enabling continuous improvement of targeting models while maintaining system reliability without proportionally increasing complexity
2Measurement precision
If the online system conducts comprehensive surveys and A/B testing to evaluate targeting models, then measurement precision improves, but loss of time increases
Solution Approach 1:
The patent applies partial action by conducting surveys and A/B testing on selected subsets of users rather than the entire user base. The system strategically samples users to achieve statistically significant results while minimizing time loss, balancing measurement precision with evaluation efficiency
Solution Approach 2:
The patent implements preliminary action by establishing benchmark metrics and evaluation frameworks in advance. This allows the system to quickly assess new targeting models against pre-defined standards without requiring extensive ad-hoc testing, thereby improving measurement precision while reducing evaluation time
Data Source
AI summary
An online system predicts, using a first targeting model, a first group of users as candidates to be in a targeting cluster, and predicts, using a second targeting model, a second group of users as candidates to be in the targeting cluster. The online system determines a first set of users that are not part of the first group of users, and a second set of users that are not part of the second group of users, and provides surveys to the first and second set of users. The online system determines a first subgroup of the first group of users and a second subgroup of the second group of users, and provides an ad preferences tool to the first subgroup and the second subgroup. The online system scores the first and second targeting models based in part on responses to the surveys and/or the ad preferences tools.