Multistage Feed Ranking With Lightweight Multi-Objective Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multistage feed ranking systems face challenges in achieving scalability, precision, and recall due to the use of less complex first pass rankers that result in reduced recall at the second pass ranker stage, especially when dealing with large volumes of feed items.
Innovation Solution
Approximate a more complex multi-objective feed item scoring model using a less complex single objective model in a multistage feed ranking system, where the first pass ranker efficiently scores a larger number of possible feed items with reduced resource consumption and high recall, while the second pass ranker optimizes precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a less complex first pass ranker is used to score feed items, then resource consumption is reduced and processing speed is improved, but recall is reduced leading to fewer relevant items reaching the second pass ranker
Solution Approach 1:
The ranking system is divided into multiple passes (first pass ranker, second pass ranker, third pass ranker), where each pass handles a subset of feed items with appropriate complexity. The first pass ranker quickly filters a large set of possible feed items to generate a candidate set, the second pass ranker applies more complex multi-objective scoring to the candidate set, and the third pass ranker performs final refinement. This segmentation allows the system to balance processing speed and recall by applying computational complexity only where necessary.
Solution Approach 2:
The first pass ranker performs preliminary filtering of feed items before they reach the second pass ranker. By pre-generating a candidate set of potentially relevant items using simpler scoring, the system prepares data in advance for the more complex multi-objective optimization, reducing the computational burden on subsequent passes while maintaining recall of relevant items.
2Measurement precision
If a complex multi-objective model is used at the first pass ranker, then precision is improved, but resource consumption and processing time increase significantly
Solution Approach 1:
Different levels of scoring complexity are applied to different subsets of feed items based on their position in the ranking pipeline. The first pass ranker uses simpler scoring for the large set of possible feed items, while the second and third pass rankers apply complex multi-objective scoring only to the smaller candidate set. This local differentiation of quality ensures high precision where it matters most while conserving resources.
Solution Approach 2:
Instead of applying full multi-objective scoring to all possible feed items, the system applies partial action by using simpler first-pass scoring for the bulk of items and reserving complex scoring for only the candidate set that has a reasonable chance of being relevant. This partial application of complex scoring significantly reduces resource consumption while maintaining overall system precision.
3Productivity
If the candidate set size is reduced to improve processing efficiency, then productivity is improved, but recall is worsened as fewer relevant items are captured
Solution Approach 1:
The system dynamically adjusts the candidate set size based on the performance and characteristics of the first pass ranker. The first pass ranker is configured to return a candidate set that is large enough to maintain high recall of relevant items while being small enough to allow efficient processing by the second pass ranker. This dynamic balancing act optimizes both productivity and recall.
Solution Approach 2:
The system uses feedback from the first pass ranking to inform the second pass ranking. By analyzing which items from the possible set made it into the candidate set and their scores, the system can adjust parameters to ensure the candidate set size maintains adequate recall while preserving processing efficiency. Feedback loops allow continuous optimization of the candidate set size.
Data Source
AI summary
Approximating a more complex multi-objective feed item scoring model using a less complex single objective feed item scoring model in a multistage feed ranking system of an online service. The disclosed techniques can facilitate multi-objective optimization for personalizing and ranking feeds including balancing personalizing a feed for viewer experience, downstream professional or social network effects, and upstream effects on content creators. The techniques can approximate the multi-objective model—that uses a rich set of machine learning features for scoring feed items at a second pass ranker in the ranking system—with the more lightweight, single objective model—that uses fewer machine learning features at a first pass ranker in the ranking system. The single objective model can more efficiently score a large set of feed items while maintaining much of the multi-objective model's richness and complexity and with high recall at the second pass ranking stage.