Graph Neural Network Incremental Learning for Recommender Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Graph Neural Network (GNN) based recommender systems take a long time to train, making it challenging to provide up-to-date recommendations as user preferences and item popularity change rapidly, necessitating a solution to reduce the time required for updating these systems.
Innovation Solution
The method involves applying a loss function to compute updated parameters for a new GNN model initialized with existing parameters, using knowledge distillation to incorporate local and global graph structures, and self-embeddings, allowing incremental updates without retraining on all existing data, thereby reducing computational resources and mitigating catastrophic forgetting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full retraining of GNN model is performed to ensure accurate recommendations, then recommendation accuracy is improved, but training time increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing node embeddings from the historical graph before the incremental update. These pre-computed embeddings serve as a foundation that reduces the computational burden during the incremental training phase, allowing the model to be updated quickly while maintaining accuracy.
Solution Approach 2:
The patent segments the training process into two distinct phases: (1) initial full training on the historical graph to obtain node embeddings, and (2) incremental update phase using only the new subgraph. This segmentation allows the computationally intensive part to be done once, with subsequent updates being much faster.
2Productivity
If incremental learning is applied to reduce training time, then training speed is improved, but catastrophic forgetting occurs
Solution Approach 1:
The patent uses node embeddings as an intermediary that captures the structural information from the historical graph. During incremental learning, these embeddings are preserved and integrated with new information from the subgraph, serving as a bridge that prevents catastrophic forgetting while enabling fast updates.
Solution Approach 2:
The patent copies the node embeddings from the historical graph and uses them as initialization for the incremental update process. This copying mechanism ensures that the structural knowledge from the historical data is preserved and can be combined with new information without losing old knowledge.
3Loss of time
If frequent model updates are performed to reflect changing user preferences, then recommendation timeliness is improved, but computational resource consumption increases
Solution Approach 1:
The patent applies partial action by performing incremental learning only on the new subgraph rather than retraining on the entire graph. This partial update approach enables frequent model updates with minimal computational resources, as only the necessary portion of the data (the new subgraph) is processed.
Data Source
AI summary
System and method for training a recommender system (RS). The RS is configured to make recommendations in respect of a bipartite graph that comprises a plurality of user nodes, a plurality of item nodes, and an observed graph topology that defines edges connecting at least some of the user nodes to some of the item nodes, the RS including an existing graph neural network (GNN) model configured by an existing set of parameters. The method includes: applying a loss function to compute an updated set of parameters for an updated GNN model that is trained with a new graph using the first set of parameters as initialization parameters, the loss function being configured to distil knowledge based on node embeddings generated by the existing GNN model in respect of an existing graph, wherein the new graph includes a plurality of user nodes and a plurality of item nodes that are also included in the existing graph; and replacing the existing GNN model of the RS with the updated GNN model.