Ranking Content Items Using Multi-Signal User Engagement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for ranking content items on mobile platforms are inadequate as they rely solely on click-based signals, which are insufficient for optimizing ranking models, and require expensive human inputs for data annotation, limiting scalability and accuracy.
Innovation Solution
A system and method for ranking content items using a plurality of user engagement signals, including click/skip, pre-click browsing time, post-click dwell time, and reformulations, to generate aggregated scores and train a ranking model, leveraging machine learning to combine and normalize these signals for optimal user satisfaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional click-based signals are used for ranking content items, then the system can determine user engagement, but the ranking model optimization is insufficient and requires expensive human inputs for data annotation
Solution Approach 1:
The system uses automated machine learning models to process user engagement signals and generate ranking optimizations without requiring human annotators. The model automatically learns from multiple signal types (clicks, skips, browsing time, dwell time, reformulations) to determine content relevance, replacing expensive manual data annotation with self-service automated processing
Solution Approach 2:
The patent transforms the ranking system by changing from relying on a single parameter (click-based signals) to utilizing multiple parameters (clicks, skips, browsing time, dwell time, reformulations). This parameter expansion enables more comprehensive automated model training, improving ranking accuracy while eliminating the need for human-labeled data
2Productivity
If a single type of user engagement signal is used, then the system can process data efficiently, but the ranking model cannot be optimized effectively
Solution Approach 1:
The system merges multiple types of user engagement signals (clicks, skips, browsing time, dwell time, reformulations) into a unified processing framework. By combining these diverse signal types, the system maintains data processing efficiency while significantly improving ranking model optimization accuracy through multi-signal analysis
Solution Approach 2:
The patent creates a universal processing framework that handles multiple types of user engagement signals through a single automated machine learning model. This multi-functional approach allows the system to efficiently process various signal types simultaneously, improving both productivity and measurement precision
3Measurement precision
If human-labeled data is used for ranking model optimization, then the model can be trained accurately, but the system cannot be scaled up
Solution Approach 1:
The system replaces human-labeled data with automated machine learning that processes user engagement signals independently. This self-service approach maintains high model training accuracy while enabling system scalability, as the automated process can handle large volumes of data without additional human resources
Solution Approach 2:
The patent substitutes the mechanical process of human data labeling with an automated computational system. The machine learning model automatically extracts features and trains on user engagement signals, replacing the manual mechanical process with an automated electronic system that scales efficiently
Data Source
AI summary
The present teaching relates to method, system, and programs for training a ranking model for ranking content items. In one example, a set of content items is obtained. A plurality types of online user activities performed with respect to the set of content items are obtained. For each of the set of content items, a plurality of user engagement scores are determined. Each of the plurality of user engagement scores is determined based on a corresponding one of the plurality types of online user activities. For each of the set of content items, an aggregated score is calculated based on the plurality of user engagement scores to generate aggregated scores. A ranking model is trained based on the aggregated scores.