Pairwise Comparison Survey for Content Quality Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional social networking systems face challenges in accurately estimating the quality of content items for user feeds, as user interactions may not reflect preferences and rating scales can be subjective and prone to noise.
Innovation Solution
Implementing a pairwise comparison method where users select between two content items in real-time surveys to determine a ranking model that assigns scores based on user preferences, using features like content type and user engagement, and adjusting scores dynamically based on user feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If user interactions are used to estimate content quality, then content quality estimation can be obtained, but the estimation is inaccurate due to subjectivity and noise in user interactions
Solution Approach 1:
The patent introduces an intermediary rating model that acts as a mediator between user interactions and content quality estimation. The rating model processes multiple user interaction signals (likes, comments, shares, watch time) through a structured framework that reduces noise and subjectivity. This intermediary layer transforms raw, unreliable user interactions into a more reliable composite quality score, resolving the contradiction between using user interactions and achieving accurate quality estimation.
2Productivity
If conventional rating scales are used for content quality estimation, then quality scores can be assigned, but the scores are subjective and prone to noise
Solution Approach 1:
The patent transforms the conventional single-dimension rating scale into a multi-parameter evaluation system. Instead of relying on a simple subjective score, the system evaluates multiple parameters including engagement metrics (likes, comments, shares), temporal metrics (watch time, interaction timing), and user profile parameters. This parameter transformation maintains estimation efficiency while significantly improving objectivity by distributing the quality assessment across multiple independent dimensions rather than a single subjective rating.
3Measurement precision
If pairwise comparison surveys are implemented in real-time, then more accurate content quality estimation is achieved, but system complexity increases
Solution Approach 1:
The patent segments the complex survey system into distinct functional modules: a survey generation module that creates pairwise comparisons, a survey delivery module that presents surveys to users in real-time, a data collection module that captures user choices, and a model training module that processes the collected data. This segmentation allows each module to be optimized independently and facilitates parallel processing, reducing overall system complexity while maintaining high measurement precision through coordinated module interactions.
4Measurement precision
If user selections are collected in real-time surveys, then noise is reduced in quality estimation, but time consumption for users increases
Solution Approach 1:
The patent implements a partial action approach where users are not required to complete exhaustive surveys for all content. Instead, the system presents a selective subset of pairwise comparisons based on user profile, content categories, and engagement patterns. Users provide partial feedback on relevant content pairs, which is sufficient for the rating model to generate accurate quality estimates. This partial survey approach maintains high measurement precision while significantly reducing user time investment compared to comprehensive survey methods.
Data Source
AI summary
Systems, methods, and non-transitory computer readable media are configured to select a first content item and a second content item to display to each user of a plurality of users in pairwise comparison in a survey associated with a content feed of the user. A selection by each user of the plurality of users of one of the first content item and the second content item can be obtained via a user interface. A ranking model configured to assign a score indicative of quality of each content item of a plurality of content items associated with content feeds based at least in part on the selections by the plurality of users can be determined.