Social Network Feed Ranking Using Human Rater Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Social networking systems face challenges in presenting relevant content to users as machine-based ranking systems may overlook human-centric factors, leading to suboptimal content interaction and engagement.
Innovation Solution
A quality-controlled and representative pool of human raters is established to provide content ratings, with consistency and representativeness scores determining the relevance of content items, allowing for the refinement of the feed ranking model to improve content relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If machine-based ranking systems are used to determine content relevance, then automation and efficiency are improved, but the ability to capture human-centric factors is reduced
Solution Approach 1:
The patent introduces human raters as intermediaries between the machine-based ranking system and the final content recommendations. These raters evaluate content using human-centric factors that machines cannot capture, such as nuanced understanding of user interests and contextual relevance. The human ratings then feed back into the machine learning model to refine future automated rankings, creating a hybrid system that combines the efficiency of automation with the precision of human judgment.
2Measurement precision
If human raters are used to provide content ratings, then content relevance accuracy is improved, but system complexity and operational overhead increase
Solution Approach 1:
The patent implements a self-service mechanism where existing social networking system users are recruited as human raters based on their profiles and demonstrated preferences. The system automatically identifies suitable raters, assigns content for evaluation, and processes their feedback without requiring external recruitment or manual coordination. This self-organizing approach reduces operational overhead while maintaining rating quality through automated matching of rater profiles to appropriate content evaluation tasks.
3Reliability
If a large pool of human raters is recruited to ensure representativeness, then rating reliability is improved, but recruitment and management costs increase
Solution Approach 1:
The patent dynamically adjusts the parameters of the rater pool based on the specific content being evaluated and the desired outcomes. Rather than maintaining a static large pool, the system modifies rater selection criteria, evaluation task assignments, and feedback weighting according to content type, user demographics, and relevance criteria. This parameter-based flexibility allows the system to achieve reliable ratings with a smaller, more strategically selected rater pool, reducing costs while maintaining or improving rating quality.
Data Source
AI summary
A social networking system builds a quality controlled and desired population-representative pool of human raters to provide ratings on content items to improve a feed ranking model used for providing its users with more relevant content. The system identifies a pool of candidate human raters for providing ratings on a feed of content items. For each candidate human rater of the pool of candidate human raters, the system presents a feed of content items based on a feed ranking model, obtains ratings on the feed of content items, and determines a score representing the consistency of the obtained ratings, the representativeness of the pool of human raters, or the relevance of the content provided by the ranking model. The system uses the computed scores to modify the ranking model used to present content to its users for improving the relevance of the presented content.