Multistage Feed Ranking System with Feature Cost Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The multistage feed ranking system faces challenges in scaling efficiently due to high computing resource usage and reduced recall at the second pass ranker, primarily because of the complexity of the multi-objective model used, which is cost-prohibitive and impractical for large-scale online services.
Innovation Solution
The system supplements feature importance scores with computing resource costs and approximates a multi-objective model using a less complex single objective model at the first pass ranker, allowing for efficient scoring of feed items while maintaining high recall at the second pass ranking stage, using techniques such as weighting user input actions and training linear regression models to balance objectives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a multi-objective model is used at the second pass ranker to balance multiple objectives (relevance, upstream effects, downstream effects), then the precision of feed item selection is improved, but the computing resource usage increases significantly
Solution Approach 1:
The patent segments the ranking process into two distinct passes: a first pass ranker that performs quick filtering of possible feed items using simpler scoring, and a second pass ranker that applies the complex multi-objective model only to the reduced candidate set. This segmentation allows the system to maintain high precision through the multi-objective model while reducing overall computing resource usage by limiting its application scope.
2Measurement precision
If a complex multi-objective model is used at the second pass ranker, then the precision of feed item selection is improved, but the device complexity increases
Solution Approach 1:
The patent divides the ranking system into two functional segments with different model complexities. The first pass ranker uses a simpler model for broad filtering, while the second pass ranker employs the complex multi-objective model only on the reduced candidate set. This segmentation manages device complexity by distributing computational tasks across two stages rather than requiring the complex model to process all possible feed items.
Solution Approach 2:
The first pass ranker acts as an intermediary between the large set of possible feed items and the second pass ranker. It filters and reduces the candidate set before passing items to the multi-objective model, thereby mediating the complexity burden and enabling the second pass ranker to maintain high precision without overwhelming system complexity.
3Speed
If the first pass ranker uses a simpler model to score feed items quickly, then the processing speed is improved, but the recall at the second pass ranker is reduced
Solution Approach 1:
The first pass ranker performs a partial scoring action by evaluating feed items using a simpler model and selecting only the top-scoring candidates. While this partial action reduces recall compared to using the full multi-objective model on all items, it maintains acceptable recall by ensuring that the simpler model still identifies sufficiently relevant items for further evaluation by the second pass ranker.
Data Source
AI summary
A feature importance score for a target machine learning feature of a target machine learning model used in a multistage feed ranking system for scoring feed items is supplemented with a feature computing resource cost. The feature computing resource cost represents the cost of using the target feature in the target model in terms of computing resources such as CPU, memory, network resources, etc. A tradeoff between feature importance and feature computing resource cost can be made to decide whether to have the target machine learning model use or not use the target machine learning feature in production, thereby improving the production multistage feed item ranking system and solving the technical problem of determining which machine learning features of a machine learning model represent the best tradeoff between feature importance and feature computing resource cost.