A collaborative filtering model design method based on graph neural network

Abstract

The present invention discloses a collaborative filtering model design method based on graph neural networks. First, a unified collaborative filtering model design framework based on graph neural networks is proposed. Based on this framework, we define a design space and evaluate it through a large number of experiments. Interesting findings are obtained, which provide insights for model design. Guided by these insights, the present invention compresses the original design space to obtain a compact space that contains a higher concentration of high-performance models. Experiments show that the compressed design space has higher quality and stronger generalization ability. This is the first attempt of the present invention to analyze the collaborative filtering design space based on graph neural networks, which not only deepens the understanding of scholars in the field of different design dimensions of the model, but also provides a new paradigm for the design of graph neural network methods in recommendation scenarios.

Description

Technical Field

[0001] The present invention relates to the field of data mining, and in particular to a collaborative filtering model design method based on graph neural network. Background Art

[0002] Collaborative filtering (CF) is a fundamental technology in modern recommendation systems. It learns user preferences by parameterizing users and items into embeddings based on interactions between users. Matrix factorization (MF) is one of the most traditional CF techniques. It projects a single-hot ID of a user / item into an embedding vector and then reconstructs the user-item interaction matrix using the dot product of the user and item embedding vectors. Deep learning has achieved tremendous success in many fields in recent years, inspiring the proposal of many CF models using neural network architectures. For example, Neural Collaborative Filtering (NCF) uses a multi-layer perceptron (MLP) as the user-item interaction function instead of the traditional dot product.

[0003] In recent years, graph neural networks (GNNs) have achieved tremendous success in the collaborative filtering (CF) field due to their ability to learn effective user / item representations. Most GNN-based collaborative filtering models adopt a message passing (or neighbor aggregation) design paradigm, where the representation of a central node is iteratively updated by aggregating the representations of its neighboring nodes, known as "messages." For example, Neural Graph Collaborative Filtering (NGCF) propagates messages based on the affinity between the graph structure and the central node, combining representations from different layers to obtain the final node representation. Lightweight Graph Collaborative Filtering (NGCF) considers nonlinearity and weight matrices useless for collaborative filtering and proposes a simple collaborative filtering model based on a graph convolutional neural network (GCN). Disentangled Graph Collaborative Filtering (DGCF) decomposes user / item representations into several components, reflecting user preferences from multiple perspectives.

[0004] In recent years, evaluative research on graph neural networks and collaborative filtering has become a new trend. For graph neural networks, Shchur et al. studied the impact of dataset segmentation on model performance and compared comprehensive evaluation results under different dataset segmentations. Lv et al. pointed out the problems in the evaluation of heterogeneous graph neural networks and proposed a comparative benchmark. Zhang et al. compared the performance of network embedding in hyperbolic space and Euclidean space. Previous work in the recommendation field also explored the issues of model reproducibility and fair evaluation. They found that a series of recently proposed complex neural network-based models were not as effective as simple and well-optimized baseline algorithms. You et al. explored the design space and task space of graph neural networks and conducted experiments to provide guidance for better graph neural network model design.

[0005] Although various collaborative filtering models based on graph neural networks have been proposed, they only design specific model architectures for specific scenarios. In addition, as an important collaborative filtering technology, collaborative filtering methods based on graph neural networks have not been fully experimentally evaluated. Summary of the Invention

[0006] To address these technical issues, this paper provides a novel collaborative filtering model design method based on graph neural networks, specifically analyzing the model design space. Unlike existing techniques that focus on comparing the performance of individual models, this paper elevates the research to propose a high-quality model design space, ensuring greater robustness and generalizability, enabling the design of optimally performing models across diverse recommendation scenarios.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A collaborative filtering model design method based on graph neural network includes the following steps:

[0009] S1. We propose a unified design framework for collaborative filtering models based on graph neural networks, which consists of four modules: initialization, graph neural network, multiple components, and interaction functions.

[0010] S2. Based on the above unified design framework, multiple design dimensions are extracted and distributed in four modules. Each design dimension contains multiple design options to establish a design space.

[0011] S3, evaluate the design space using controlled random search;

[0012] S4. Compress the design space by reducing the design choices of the design dimensions to produce a model with good performance.

[0013] Furthermore, in step S1, the initialization module projects the unique IDs of users and products into dense real-valued embedding vectors through the embedding matrix ID search, i.e., e u =lookup(ID(u)) and e i =lookup(ID(i)), where e u and e i are the initial embeddings of user u and item i, respectively.

[0014] Furthermore, in step S1, the GNN inputs the initial embeddings of users and products into the GNN, updates the embedding representation through propagation through the GNN layers, and combines the outputs of each GNN layer to obtain the representation. The process is as follows:

[0015]

[0016]

[0017]

[0018]

[0019] in, and They represent the updated representations of user u and item i after being processed by the l-layer graph neural network, and m(·) represents the message flow from encoding item i to user u. The message function, σ is the activation function, are the neighbors of user u, f(·) is the neighbor information aggregation method, and g(·) is the layer aggregation function that aggregates L+1 representations after L layers of propagation.

[0020] Furthermore, in step S1, the multi-component module decomposes the user and product representations into multiple components and performs K independent embedding processes to obtain K components represented as h u，1 , h u，2 ,…,h u，K , and combine them to get the final representation as follows:

[0021] h u =c(h u，1 , h u，2 ,…,h u，K )

[0022] where c(·) is the component aggregation function.

[0023] Furthermore, the interactive function module in step S1 performs user-item matching and predicts the rating of the user-item pair to reflect the user's preference, as shown below:

[0024]

[0025] Among them, p(·) is the interaction function, and the predicted rating of the user-item pair (u, i) is

[0026] Furthermore, the design dimensions in step S2 are: initial embedding dimension d, message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·), number of components K, component aggregation function c(·) and interaction function p(·). The initial embedding dimension d is distributed in the initialization module, the message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·) and number of components K are distributed in the graph neural network module, the component aggregation function c(·) is distributed in the multi-component module, and the interaction function p(·) is distributed in the interaction function module.

[0027] Furthermore, the design choices of the initial embedding dimension d include 64, 128 and 256, the design choices of the message function m(·) include Identity and Hadamard, the design choices of the aggregation function f(·) include None, GCN, GAT, GIN and GraphSAGE, the design choices of the activation function σ(·) include Identity, Sigmoid, Tanh, ReLU, PReLU and LeakyReLU, the design choices of the number of layers L include 1, 2, 3 and 4, the design choices of the inter-layer aggregation function g(·) include Stack, Concat, Sum and Mean, the design choices of the number of components K include 1, 2, 3 and 4, the design choices of the component aggregation function c(·) include Concat, Mean and Att, and the design choices of the interaction function p(·) include Dot Product, Concat+MLP and Sum+MLP.

[0028] Furthermore, the evaluation method of step S3 is as follows: the Cartesian product of the design dimension and the data set is defined as a configuration space, and a controlled random search is performed in the configuration space to obtain S experimental configurations; at the same time, all other dimensions are controlled unchanged to obtain S groups, each group has the same number of experimental configurations as the number of design choices, and the only difference between them is the design choice of the design dimension to be evaluated; in each group, the design choices are sorted by performance. If the performance difference is less than 0.0001, the ranking is equal, and the different design choices are ranked averagely in all S groups.

[0029] Furthermore, the design dimensions in step S4 are: initial embedding dimension d, message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·), number of components K, component aggregation function c(·) and interaction function h(·).

[0030] Furthermore, the design choice of the initial embedding dimension d is 64, the design choices of the message function m(·) are Identity and Hadamard, the design choices of the aggregation function f(·) are None and GraphSAGE, the design choices of the activation function σ(·) are Identity, Sigmoid and ReLU, the design choices of the number of layers L are 1 and 2, the design choice of the inter-layer aggregation function g(·) is Mean, the design choices of the number of components K are 1 and 4, the design choice of the component aggregation function c(·) is Att, and the design choices of the interaction function h(·) are Concat+MLP and Sum+MLP.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] This paper presents a method for designing collaborative filtering models based on graph neural networks. It first proposes a unified framework for designing collaborative filtering models based on graph neural networks. Within this framework, we define a design space and evaluate it through extensive experiments, obtaining interesting findings that provide insights for model design. Guided by these insights, we compress the original design space to obtain a more compact space containing a higher concentration of high-performance models. Experiments demonstrate that this compressed design space exhibits higher quality and stronger generalization capabilities.

[0033] In summary, this invention is the first to attempt to analyze the collaborative filtering design space based on graph neural networks. This not only deepens the understanding of scholars in the field on the different design dimensions of the model, but also provides a new paradigm for the design of graph neural network methods in recommendation scenarios. Unlike existing work that focuses on comparing the performance of a single model, this invention elevates the research level to propose a high-quality model design space, which contains a higher concentration of high-performance models, ensuring stronger robustness and generalization, and can work well under different recommendation settings. Faced with more recommendation domain-specific challenges such as the sparsity and diversity of user interests, the present invention can customize the evaluation strategy accordingly, conduct more in-depth analysis, draw more valuable conclusions for the field, and promote the development of field research. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0035] Figure 1 A unified design framework for collaborative filtering based on graph neural networks provided by the embodiments of the present invention;

[0036] Figure 2 This section provides a ranking analysis of the nine design dimensions provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0038] The collaborative filtering model design method based on graph neural network of the present invention comprises the following steps:

[0039] S1. A unified design framework for collaborative filtering models based on graph neural networks is proposed. The framework includes four modules: initialization, graph neural networks, multiple components, and interaction functions. The overall framework of the model is as follows: Figure 1 shown.

[0040] Taking the representation update process on the user side as an example, we explain the framework as follows.

[0041] a) Initialize the embedding matrix ID lookup to project the user / item’s unique ID into a dense real-valued embedding vector, i.e., e u =lookup(ID(u)),e i =lookup(ID(i)), where e u (e i ) is the initial embedding of user u (item i). This is a standard approach to obtain embeddings when no other features are available, such as user profiles or item attributes.

[0042] b) The GNN inputs the initial embedding of the user / item into the GNN, updates the embedding representation through propagation through the GNN layers, and combines the outputs of each GNN layer (including the initial embedding) to obtain the representation. The process is as follows:

[0043]

[0044]

[0045]

[0046]

[0047] in, represents the updated representation of user u (item i) after the l-layer graph neural network, and m(·) represents the message flow from encoding item i to user u. The message function, σ is the activation function, are the neighbors of user u, f(·) is the neighbor information aggregation method, and g(·) is the layer aggregation function that aggregates L+1 representations after L layers of propagation.

[0048] c) Multi-component By decomposing the user / item representation into multiple components, we learn how to better model different user interests from different aspects. Specifically, we perform K independent embedding processes according to the formula given in b), and obtain K components represented as h u，1 , h u，2 ,…,h u，K , and combine them to get the final representation as follows:

[0049] h u =c(h u，1 , h u，2 ,…,h u，K ),

[0050] where c(·) is the component aggregation function.

[0051] d) The interaction function performs user-item matching and predicts the rating of the user-item pair to reflect the user preference, as shown below:

[0052]

[0053] Among them, p(·) is the interaction function, and the predicted rating of the user-item pair (u, i) is

[0054] S2. Based on the above unified design framework, nine design dimensions are extracted and distributed in four modules, such as Figure 1 As shown in Table 1, each design dimension contains multiple design options to establish a design space.

[0055] Table 1 Design dimension selection

[0056]

[0057]

[0058] The model architectures proposed in current work differ in their design. For example, LightGCN removes nonlinear activation from their proposed architecture, arguing that it is not beneficial for collaborative filtering, while it remains present in many other works. Different combinations of design choices lead to different model instances with varying recommendation performance. Therefore, when designing a model, careful consideration must be given to the specific design dimensions chosen.

[0059] To explore the impact of different design dimensions, we propose a research design space, defined as the Cartesian product of design dimensions, encompassing a set of model instances. Note that our goal is not to present the broadest design space possible, but rather to help understand the impact of different design dimensions for collaborative filtering with graph neural networks and to gain insights into designing well-performing models. Indeed, the dimensionality and scope of design can be naturally expanded by including more options.

[0060] Some important design dimensions are explained below, while other dimensions can be naturally understood based on Table 1.

[0061] Message function m(·). In the relevant literature, the most common approach is to directly use Some work suggests that the interaction between the source and target nodes should also be encoded into the message. Therefore, we also consider The above two design choices are denoted as Identity and Hadamard respectively.

[0062] Aggregation function f(·). For this design dimension, we consider four common and effective graph neural network methods as design options: GCN, GAT, GIN, and GraphSAGE. In particular, we generalize the design dimension to include the option of None, which means that no graph information is utilized. The user / item representation is refined through MLP to expand the capacity of the design space to include non-graph neural network models.

[0063] Inter-layer aggregation function g(·). Stack represents the direct stacking of multiple graph neural network layers, using the output of the last layer to obtain the corresponding representation of each component. Because some work has found that the output of intermediate layers is also effective for tasks, we will investigate three other inter-layer aggregation methods: Concat, Sum, and Mean.

[0064] The component aggregation function c(·). A straightforward strategy used in existing work (e.g., DGCF) is to concatenate the representations, denoted as Concat. Alternatively, an attention mechanism can be used, denoted as Att. We add an additional design choice, Mean.

[0065] The interaction function p(·) is a simple and effective choice, computing the dot product of the user and item representations, denoted as DotProduct. Neural networks can also be used to learn the interaction function. The user and item representations are first concatenated or summed, and then fed into an MLP for prediction, denoted as Concat+MLP and Sum+MLP, respectively.

[0066] S3. Evaluate the design space using controlled random search.

[0067] After defining the complete design space, a grid search can be performed. This involves exploring the performance of the proposed model for each possible combination of dimensions in the design space, thereby gaining a complete understanding of the impact of different choices in each design dimension on model performance. However, with over 100,000 model architectures in the design space, conducting a full grid search to evaluate each design dimension is prohibitively expensive and time-consuming, making it impractical in a design space with complex dimensions. To address this issue, we employ controlled random search as a design space evaluation strategy.

[0068] To distribute the evaluation across different datasets, we first define a configuration space, the Cartesian product of the design dimensions and the dataset, and perform a controlled random search in the configuration space to obtain experimental configurations. Suppose we want to evaluate the message function m(·). We first draw S experimental configurations by randomly searching the configuration space, all with m(·) = Identity. We then obtain another S configurations by setting m(·) = Hadamard, while holding all other dimensions constant. We now have S groups, each with two experimental configurations that differ only in m(·). Within each group, the two design choices from {Identity, Hadamard} are ranked by performance, with the ranking considered tied if the performance difference is less than 0.0001.

[0069] The average ranking of different options in all S groups is shown in the bar chart. The ranking analysis of the nine design dimensions (the lower the better) is shown in the figure below. Figure 2 In our experiments, we set S = 100, which reduces the number of experiments from 103680 to 3400, a reduction of more than 30 times.

[0070] It should be noted that the purpose of this invention is not to find a single optimal model from all these configurations, but to explore whether there are findings that can enrich the understanding of design dimensions and help efficiently design high-performance graph neural network-based collaborative filtering models in different recommendation scenarios.

[0071] Some key experimental results are listed below:

[0072] GAT and GraphSAGE outperform the other options. Interestingly, None performs comparably to other graph neural network-based aggregation methods, suggesting that in some scenarios, simple MF or MLP-based collaborative filtering methods can achieve comparable or even better performance than graph neural network models. This interesting finding reminds us that graph information is not always beneficial for collaborative filtering.

[0073] Sigmoid performs significantly better among all six activation functions.

[0074] When considering multiple components, setting the number of components to 4 is more advantageous, which is consistent with the findings of previous work that users’ interests are different in different aspects. It is better to combine the representation of different components with the Att mechanism.

[0075] Using neural interaction functions is better than using dot products.

[0076] These findings not only enrich our understanding of the impact of different design dimensions but also provide valuable insights for effectively designing high-performance models. Specifically, we observe a certain amount of redundancy in the design space. For example, the initial embedding dimension can be fixed at 64, as it significantly outperforms the other two options. This suggests that further pruning of the original design space can be performed to improve its quality, thereby enhancing the efficiency of the search for high-performance models.

[0077] S4. Compress the design space by reducing the design choices of the design dimensions to produce a model with good performance.

[0078] Based on the above findings, we compress the common design space by narrowing the choices in the design dimensions. The motivation is to retain only favorable design choices that are empirically more likely to produce well-performing models. As a result, the compressed design space contains a higher concentration of high-performing models, which will facilitate model search.

[0079] The design dimension selection in the compressed design space according to the embodiment of the present invention is shown in Table 2.

[0080] Table 2 Design dimension selection in compressed design space

[0081]

[0082] Below, we briefly explain the compression process from two perspectives. Regarding aggregation, we chose GraphSAGE as a representative graph neural network aggregator because its training overhead is lower than GAT. None is also retained, allowing the compressed design space to include non-graph neural network models. Regarding activation functions, to better accommodate linear and nonlinear models, we retained two well-recommended nonlinear functions and Identity.

[0083] After compression, only 96 candidate models remain in the design space, compared to 103,680 in the original space. This represents a three-order-of-magnitude reduction (1,080x) in the size of the design space. The advantage of the compressed design space over the original design space is that it simplifies the combination of design dimensions by eliminating suboptimal choices, thereby containing a higher concentration of high-performing models that are well-suited for efficient search across scenarios. The compressed design space consistently demonstrates high quality across different recommendation settings, demonstrating its strong generalization capabilities.

[0084] This paper proposes a unified framework for collaborative filtering model design based on graph neural networks. Based on this framework, we define a design space and evaluate it through extensive experiments, obtaining interesting findings that provide insights for model design. Guided by these insights, we compress the original design space to obtain a compact space containing a higher concentration of high-performing models. Experiments demonstrate that this compressed design space exhibits higher quality and stronger generalization capabilities.

[0085] In summary, this invention is the first to attempt to analyze the collaborative filtering design space based on graph neural networks. This not only deepens the understanding of scholars in the field on the different design dimensions of the model, but also provides a new paradigm for the design of graph neural network methods in recommendation scenarios. Unlike existing work that focuses on comparing the performance of a single model, this invention elevates the research level to propose a high-quality model design space, which contains a higher concentration of high-performance models, ensuring stronger robustness and generalization, and can work well under different recommendation settings. Faced with more recommendation domain-specific challenges such as the sparsity and diversity of user interests, the present invention can customize the evaluation strategy accordingly, conduct more in-depth analysis, draw more valuable conclusions for the field, and promote the development of field research.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collaborative filtering model design method based on graph neural network, characterized in that: The following steps are involved: S1. We propose a unified design framework for collaborative filtering models based on graph neural networks, which consists of four modules: initialization, graph neural network, multiple components, and interaction functions. The initialization module projects the unique IDs of users and products into dense real-valued embedding vectors through the embedding matrix ID search, i.e., e u =lookup(ID(u)) and e i =look up(ID(i)), where e u and e i are the initial embeddings of user u and item i respectively; The graph neural network inputs the initial embeddings of users and products into the graph neural network, updates the embedding representation through propagation through the graph neural network layers, and combines the outputs of each graph neural network layer to obtain the representation. The process is as follows: in, and They represent the updated representations of user u and item i after being processed by the l-layer graph neural network, and m(·) represents the message flow from encoding item i to user u. The message function, σ is the activation function, is the neighbor of user u, f(·) is the neighbor information aggregation method, g(·) is the layer aggregation function that aggregates L+1 representations after L layers of propagation; The multi-component module decomposes the user and product representations into multiple components and performs K independent embedding processes to obtain K components represented as h u,1 ,h u,2 ,…,h u,K , and combine them to get the final representation as follows: h u =c(h u,1 ,h u,2 ,…,h u,K Where c(·) is the component aggregation function; The interaction function module performs user-item matching and predicts the rating of the user-item pair to reflect the user's preference, as shown below: Among them, p(·) is the interaction function, and the predicted rating of the user-item pair (u,i) is S2. Based on the above unified design framework, multiple design dimensions are extracted and distributed in four modules. Each design dimension contains multiple design options to establish a design space. S3, evaluate the design space using controlled random search; S4. Compress the design space by reducing the design choices of the design dimensions to produce a model with good performance.

2. The collaborative filtering model design method based on graph neural network according to claim 1 is characterized in that: The design dimensions in step S2 are: initial embedding dimension d, message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·), number of components K, component aggregation function c(·) and interaction function p(·). The initial embedding dimension d is distributed in the initialization module, the message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·) and number of components K are distributed in the graph neural network module, the component aggregation function c(·) is distributed in the multi-component module, and the interaction function p(·) is distributed in the interaction function module.

3. The collaborative filtering model design method based on graph neural network according to claim 2 is characterized in that: Design choices for the initial embedding dimension d include 64, 128, and 256, design choices for the message function m(·) include Identity and Hadamard, design choices for the aggregation function f(·) include None, GCN, GAT, GIN, and GraphSAGE, design choices for the activation function σ(·) include Identity, Sigmoid, Tanh, ReLU, PReLU, and LeakyReLU, design choices for the number of layers L include 1, 2, 3, and 4, design choices for the inter-layer aggregation function g(·) include Stack, Concat, Sum, and Mean, design choices for the number of components K include 1, 2, 3, and 4, design choices for the component aggregation function c(·) include Concat, Mean, and Att, and design choices for the interaction function p(·) include DotProduct, Concat+MLP, and Sum+MLP.

4. The collaborative filtering model design method based on graph neural network according to claim 1 is characterized in that: The evaluation method in step S3 is as follows: define the Cartesian product of the design dimension and the dataset as a configuration space, and perform a controlled random search in the configuration space to obtain S experimental configurations; at the same time, hold all other dimensions constant to obtain S groups, each with the same number of experimental configurations as the number of design choices, differing only in the design choices of the design dimension to be evaluated; within each group, the design choices are ranked by performance, and if the performance difference is less than 0.0001, the ranking is considered equal, and the different design choices are averaged across all S groups.

5. The collaborative filtering model design method based on graph neural network according to claim 1 is characterized in that: The design dimensions in step S4 are: initial embedding dimension d, message function m(·), aggregation function f(·), activation function σ(·), number of layers L, inter-layer aggregation function g(·), number of components K, component aggregation function c(·) and interaction function h(·).

6. The collaborative filtering model design method based on graph neural network according to claim 5 is characterized in that: The design choices for the initial embedding dimension d are 64, the design choices for the message function m(·) are Identity and Hadamard, the design choices for the aggregation function f(·) are None and GraphSAGE, the design choices for the activation function σ(·) are Identity, Sigmoid and ReLU, the design choices for the number of layers L are 1 and 2, the design choice for the inter-layer aggregation function g(·) is Mean, the design choices for the number of components K are 1 and 4, the design choice for the component aggregation function c(·) is Att, and the design choices for the interaction function h(·) are Concat+MLP and Sum+MLP.

Images