Sequential Recommender Using Actor-Critic and Negative Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional recommendation systems fail to flexibly account for dynamic user preferences and item relationships, leading to inaccurate and inefficient digital recommendations.
Innovation Solution
A sequential recommender system utilizing reinforcement learning and collaborative filtering, incorporating an actor-critic framework to adapt to client device behaviors, captures dynamic features and item relationships, and utilizes all interaction data, including negative feedback, to generate accurate recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional recommendation systems use static user preferences and item attributes, then the system structure is simple, but the recommendations become inaccurate and cannot adapt to dynamic user preferences
Solution Approach 1:
The patent implements dynamic user preference modeling by continuously updating user profiles based on recent interactions and temporal patterns. The system transitions from static to dynamic preference capture through time-aware recommendation models that adapt to changing user interests, thereby improving recommendation accuracy while managing complexity through structured temporal modeling approaches.
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing historical interaction data, building embedding spaces for items and users before generating recommendations. This includes pre-computing item relationships and user preferences from historical data, which then enables accurate real-time recommendations without requiring complex computations at recommendation time.
2Measurement precision
If conventional systems ignore negative feedback, then the processing is simpler, but the recommendations fail to accurately reflect user preferences
Solution Approach 1:
The patent implements comprehensive feedback mechanisms that incorporate both positive and negative user interactions into the recommendation model. Negative feedback is systematically processed to refine user preference models, allowing the system to learn from user disinterest and adjust recommendations accordingly. This feedback loop continuously improves measurement precision of user preferences.
Solution Approach 2:
The system converts negative feedback, which could be considered harmful or useless data, into beneficial information for improving recommendation accuracy. By treating negative interactions as valuable signals about user preferences, the system refines its understanding of what users do not want, thereby improving overall preference measurement accuracy.
3Productivity
If the system searches through all possible items to generate recommendations, then the recommendations are more comprehensive, but the computational resources consumed increase significantly
Solution Approach 1:
The patent segments the recommendation search space by organizing items into structured embedding spaces and using pre-computed item relationships. Instead of searching all possible items, the system divides the search into manageable segments based on user preferences, item categories, and relationship graphs, significantly reducing computational requirements while maintaining comprehensive coverage.
Solution Approach 2:
The system introduces intermediary structures such as embedding spaces, item graphs, and preference models that mediate between the user and the full item catalog. These intermediaries enable efficient navigation and filtering of large item spaces without requiring exhaustive searches, reducing computational resource consumption while maintaining recommendation quality.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize collaborative filtering and a reinforcement learning model having an actor-critic framework to provide digital content items across client devices. In particular, in one or more embodiments, the disclosed systems monitor interactions of a client device with one or more digital content items to generate item embeddings (e.g., utilizing a collaborative filtering model). The disclosed systems further utilize a reinforcement learning model to generate a recommendation (e.g., determine one or more additional digital content items to provide to the client device) based on the user interactions. In some implementations, the disclosed systems utilize the reinforcement learning model to analyze every negative and positive interaction observed when generating the recommendation. Further, the disclosed systems utilize the reinforcement learning model to analyze item embeddings, which encode the relationships among the digital content items, when generating the recommendation.