MoE Prediction Model for Multi-Task Recommendation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recommender systems face challenges in scalability and efficiency when handling multiple recommendation tasks simultaneously, particularly due to rapid degradation in training and inference speeds as the number of tasks increases, and the complexity of coordinating multiple single-task models.
Innovation Solution
The implementation of a mixture-of-experts (MoE) prediction model framework using a multi-task learning scheme, which enables sharing of information across different tasks to improve recommendation accuracy. This framework employs a sparse MoE architecture that allows for dynamic task-sentence routing to relevant experts, optimizing recommendations based on performance metrics specified in task sentences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple single-task models are coordinated to handle multiple recommendation tasks, then recommendation accuracy for each task can be maintained, but device complexity and coordination overhead increase significantly
Solution Approach 1:
The patent combines multiple single-task recommendation models into a unified multi-task learning framework. Different recommendation tasks (e.g., click-through rate prediction, conversion rate prediction, dwell time prediction) are integrated into a single model architecture that shares common layers and parameters, eliminating the need to coordinate multiple separate models while maintaining specialized capabilities for each task through task-specific output layers.
Solution Approach 2:
The multi-task recommendation model is designed with universal components that serve multiple functions. The shared embedding layers, attention mechanisms, and feature extraction modules process inputs for all recommendation tasks simultaneously, allowing a single model to perform multiple recommendation functions without requiring separate specialized models for each task type.
2Adaptability or versatility
If the number of recommendation tasks increases to improve system versatility, then adaptability to different user needs improves, but training and inference speeds degrade rapidly
Solution Approach 1:
The patent segments the multi-task recommendation model into hierarchical components: shared bottom layers for common feature extraction, intermediate task-specific branches for different recommendation objectives, and specialized output layers for each task. This segmentation allows the model to handle multiple task categories while maintaining efficient computation by processing common features once and reusing them across tasks rather than duplicating processing for each task.
Solution Approach 2:
The model employs dynamic task routing mechanisms that adaptively activate only the relevant task branches based on input characteristics and current system state. This dynamic activation allows the system to maintain high versatility by having multiple task capabilities available while preserving training and inference speed by selectively engaging only the necessary task pathways for each prediction request.
Data Source
AI summary
The present teaching relates to recommendation. Current event information and historic sequence data are received. The former characterizes a current event involving a user and user's interactions with a user interface (UI). The latter includes UIs and corresponding user interactions thereon with corresponding performance data. A task sentence is created with multiple tokens, each of which corresponds to a task. The current event information, the historic sequence data, and the task sentence are used for predicting a next item to be recommended via a mixture of expert (MoE) prediction model, trained via multi-task learning. The next item is recommended to the user on the UI.