Algorithm Selection via Predicted Click-Through Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing content delivery systems struggle to accurately identify and provide relevant content to users based solely on user interactions, as they often rely on actual click-through rates, which are limited by competition in content auctions and do not account for user behavior effectively.
Innovation Solution
A method and system that select an algorithm for generating models to identify similar user identifiers by computing and comparing weighted average predicted click-through rates, allowing for a more accurate evaluation of content relevance without the need for actual clicks, using online behavior data to determine user similarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If actual click-through rates are used to evaluate content relevance, then content delivery can be optimized based on real user responses, but the evaluation is limited by competition in content auctions and does not account for user behavior effectively
Solution Approach 1:
The system performs preliminary evaluation of content relevance using predicted click-through rates before actual content delivery and user interaction occurs. By pre-evaluating which algorithms and content combinations are most likely to succeed based on historical data and machine learning models, the system can optimize content delivery strategies in advance, reducing the need to wait for actual user clicks and auction outcomes.
Solution Approach 2:
The patent introduces predicted click-through rates as an intermediary metric between content creation and actual user interaction. Instead of directly relying on actual click-through rates that are contaminated by auction competition, the system uses predicted CTRs as a mediator to evaluate and select content algorithms, thereby obtaining a cleaner signal of content relevance that is not distorted by competitive bidding.
2Loss of time
If predicted click-through rates are used to evaluate algorithms, then content relevance can be assessed without actual clicks, but the system requires complex machine learning models and historical data processing
Solution Approach 1:
The system uses historical data and existing machine learning models to self-generate predicted click-through rates without requiring external intervention or manual analysis. The machine learning models automatically process historical user behavior data and content performance metrics to produce predictions, allowing the system to evaluate content relevance autonomously based on patterns learned from past data.
Solution Approach 2:
The patent transforms the evaluation parameter from actual click-through rates (which require real-time user interaction and auction completion) to predicted click-through rates (which can be computed from historical data and model parameters). This parameter transformation enables offline evaluation and algorithm selection without waiting for actual user behavior, significantly reducing evaluation time while using pre-computed model outputs.
3Measurement precision
If multiple algorithms are used to generate user identifier models, then content targeting accuracy can be improved, but the complexity of selecting and managing algorithms increases
Solution Approach 1:
The system implements a feedback mechanism where predicted click-through rates are used to evaluate and rank different algorithms, and the performance information feeds back into the algorithm selection process. By continuously monitoring which algorithms produce content with highest predicted CTRs and using this feedback to refine future algorithm choices, the system can automatically optimize which models to deploy without manual intervention.
Solution Approach 2:
The patent introduces weighted average predicted click-through rates as a composite evaluation parameter that aggregates performance across multiple algorithms and content items. This parameter transformation simplifies the comparison of multiple algorithms by converting their individual performance metrics into a single comparable metric, making algorithm selection and management more straightforward despite having multiple underlying models.
Data Source
AI summary
A computerized method, system for, and computer-readable medium operable to select an algorithm for generating models configured to identify similar user identifiers. A first plurality of models generated by a first algorithm is received. A plurality of lists of similar user identifiers is generated. User queries associated with user identifiers on the plurality of lists of similar user identifiers are identified. Predicted click-through rates for the user queries is received. An average predicted click-through rate is computed for each model based on the predicted click-through rates. A weighted average predicted click-through rate associated with the first plurality of models is computed. The weighted average predicted click-through rate for the first plurality of models can be compared to a weighted average predicted click-through rate for a second plurality of models generated by a second algorithm. The algorithm for generating models is selected based on the comparison.