Content Rating System with Diversity Enforcement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Online systems face challenges in obtaining accurate user ratings for content items due to diversity rules that limit the number of times a content item can be presented to a user within a time interval, which can bias user ratings and impair model training for selecting relevant content.

Innovation Solution

The online system modifies the set of content items by replacing previously presented items within the specified time interval with alternative items that have not been shown to the user, ensuring that ratings are collected for content items not previously presented, thereby allowing for more accurate user feedback while adhering to diversity rules.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If diversity rules are enforced to limit the number of times a content item is presented to a user within a time interval, then user engagement and content diversity are improved, but rating accuracy is worsened because users may not provide accurate ratings for content items they have already seen

Engineering Contradiction:

Improverating accuracyVSAvoidcontent presentation limitation

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The system performs preliminary actions by tracking which content items have been presented to each user within the time interval before the rating collection process. This pre-tracking enables the system to identify and exclude previously presented content items from the rating collection, ensuring that only new content items are rated, thereby maintaining rating accuracy while adhering to diversity rules.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts or removes previously presented content items from the set of content items that are eligible for rating collection. By taking out content items that have already been shown to the user within the time interval, the system ensures that only content items the user has not seen are included in the rating process, preventing biased ratings while maintaining diversity rule compliance.

Inventive Principle:

Principle #2Taking out (Extraction)

2Quantity of substance

If content items are presented to users multiple times within a time interval to obtain sufficient ratings, then model training data is improved, but diversity rules are violated and user engagement is reduced

Engineering Contradiction:

Improvenumber of ratings collectedVSAvoidmodel training quality

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system performs preliminary tracking of presented content items before the rating collection process. This pre-action enables the system to identify content items that have been presented within the time interval and exclude them from the rating collection, ensuring that sufficient ratings are collected from diverse content items without violating diversity rules or reducing user engagement.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system introduces an intermediary mechanism that tracks and manages the relationship between content items, users, and ratings. This intermediary tracking mechanism ensures that content items are presented and rated only once within the time interval, mediating between the need for sufficient rating data and the requirement to maintain diversity and user engagement.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Quantity of substance

If the same content items are presented to multiple users to obtain ratings, then rating data diversity is improved, but user engagement is reduced due to repeated content exposure

Engineering Contradiction:

Improverating data diversityVSAvoiduser engagement

Core Design Contradiction:

Quantity of substanceVSEase of operation

Solution Approach 1:

The system performs preliminary tracking of which content items have been presented to each user within the time interval. This pre-action enables the system to select content items for rating collection that have not been previously shown to the user, ensuring that rating data diversity is maintained while avoiding repeated content exposure that would reduce user engagement.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system applies local quality by tailoring the content presentation and rating collection process to individual user characteristics and viewing history. By considering which content items have been presented to each specific user within the time interval, the system ensures that each user receives diverse content items that they have not seen before, maintaining both rating data diversity and user engagement.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10212468B1Obtaining ratings for content items from a group of users while limiting a number of times the content items are presented to the group of users within a time interval

Publication Date: 2019.02.19 META PLATFORMS INC

AI summary

An online system presents content items to a group of users who provide ratings for the content items. Based on ratings received from various users of the group, the online system generates data describing presentation of the content items to users of the group. Because of a limited number of users in the group, the online system enforces rules that limit the ability to show content items to users of the group within a time interval. Accordingly, when a set of content items are selected for presentation to a user of the group, the online system replaces content items of the set that were previously shown to the user within the time interval with alternative content items. The online system also retrieves a previously received rating for a content item replaced by an alternative content item to use along with ratings received for content items of the set.