Deduplicating Online Recommendations via Clustering and Engagement Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Online concierge systems face the challenge of noisy 'Buy It Again' recommendations due to similar items in a user's purchase history, leading to cluttered and ineffective recommendations.
Innovation Solution
The system employs a trained computer model and clustering to deduplicate similar items, predicting engagement likelihood for each item in clusters, and selecting representative items for recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the system recommends all previously purchased items to users, then the quantity of recommendations increases, but the quality and relevance of recommendations deteriorates due to noise from similar items
Solution Approach 1:
The system segments the purchase history into multiple clusters based on item similarity. Each cluster groups together similar items (e.g., different types of yogurt), and the system selects representative items from each cluster for recommendation. This segmentation prevents overwhelming users with multiple similar recommendations while maintaining comprehensive coverage of purchase categories.
Solution Approach 2:
The system extracts and removes duplicate or highly similar items from the recommendation list by clustering. For each cluster of similar items, only one representative item is selected for recommendation, effectively taking out redundant recommendations that would otherwise noise the recommendation list.
2Reliability
If the system clusters items based on similarity, then the noise in recommendations is reduced, but the complexity of the recommendation system increases
Solution Approach 1:
The system uses unsupervised learning clustering algorithms that automatically group items based on their features without requiring manual configuration or supervision. The clustering process self-organizes the purchase history data, and the system automatically selects representative items from each cluster, reducing the need for complex manual rule-based systems.
Solution Approach 2:
The system changes the parameter of item representation from individual item features to cluster-level representations. By transforming the recommendation problem from selecting individual items to selecting representative items from clusters, the system simplifies the recommendation space while maintaining quality.
3Measurement precision
If the system uses a trained computer model to predict engagement likelihood, then the precision of recommendation selection improves, but the computational resources and time required increase
Solution Approach 1:
The system performs preliminary clustering of the entire purchase history before applying the engagement prediction model. This pre-processing step groups similar items together, so the model only needs to evaluate one representative item per cluster rather than every individual item, significantly reducing computational resources while maintaining prediction accuracy for the final recommendations.
Data Source
AI summary
A trained computer model to identify a list of representative previously purchased items for recommendation to a user of an online system. The online system clusters, based on a similarity score for each pair of items, a set of previously purchased items into multiple clusters. The online system accesses a computer model trained to predict a likelihood of engagement by the user for each item in each cluster, and applies the computer model to predict, based on one or more features of each item, the likelihood of engagement for each item in each cluster. The online system generates, based on the likelihood of engagement, a score for each item in each cluster. The online system selects, based on the score for each item, a representative item from each cluster. The online system causes a device associated with the user to display the representative item from each cluster.