A recommendation system method based on NSGC-GRU integrated model
By adopting the NSGC-GRU integrated model in the recommendation system, the problem of excessive smoothing of GCN and insufficient utilization of timing information is solved, more accurate personalized recommendations are achieved, and recall and performance are improved.
Patent Information
- Application Number
- CN202210388268.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-13
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2042-04-13
AI Technical Summary
The existing GCN-based recommendation system has problems with excessive smoothing, resulting in user/product embedding becoming indistinguishable, recommendation performance degrades, and timing information is not fully utilized.
The NSGC-GRU integration model is adopted to eliminate nonlinearity in GCN, simplify the propagation of user or product embedded representations, and combine the GRU model to perform timing prediction to improve recommendation accuracy.
By combining neighbor aggregation and timing prediction, the accuracy of recommendation prediction is improved, and more accurate personalized recommendations are provided, which improves recall and normalized loss cumulative gain performance.
Smart Images
Figure CN114817758B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of recommendation systems, and in particular to a recommendation system method based on an NSGC-GRU integrated model. Background Art
[0002] With the rapid development of mobile Internet, people have entered the era of information explosion. Currently, there are more and more platforms providing services through the Internet, and the corresponding types of services, such as shopping, video, etc., are emerging in an endless stream. Therefore, it is particularly important to effectively provide personalized services to users. In order to reduce the information overload on this network, recommendation systems are widely used in personalized information filtering. The core of the recommendation system is to predict whether users interact with products, such as clicks, ratings, purchases, and other forms of interaction. Collaborative filtering (CF) uses past interactions between users and products to achieve predictions, which is still the basic task for achieving effective personalized recommendations. Therefore, the recommendation system can recommend one or a series of unobserved products that the user likes.
[0003] GCN-based models have achieved great success in recommendation due to their powerful ability to learn representations from non-Euclidean structures. At its core, iteratively aggregates feature information from local graph neighbors, which has proven to be an effective way to extract additional information from graph structures, thereby improving representation learning for users and items and alleviating the sparsity problem. However, it is now also recognized that GCN has an over-smoothing problem, because the graph convolution operation is actually a special graph Laplacian smoothing, which makes the node representations indistinguishable after multiple layers of graph convolution. Therefore, most current GCN-based models achieve peak performance by stacking a few layers (2 or 3 layers), and further increasing the depth will lead to a decrease in performance. In the field of recommendation, due to the over-smoothing effect, when more layers are stacked in NGCF, the user / item embeddings become more similar. In other words, the preferences of different users become homogenized, resulting in a decrease in recommendation performance. At the same time, LightGCN [He, 2021] points out that feature transformation and non-linear activation have no positive effect (or even no negative effect), so too many feature transformations and non-linear activations will only increase the size and complexity of the calculation, increase the computational time cost without getting better performance. However, the current GCN-based recommendation does not fully explore the temporal information, which plays an important role in the importance weight distribution of different neighbors. User preferences may change. For example, a user may no longer like a product that he originally liked because he encountered a better product. Another example is that a user did not like a product but took related actions because the sales and quality of the product increased significantly. Summary of the invention
[0004] The present invention proposes a recommendation system method based on the NSGC-GRU integrated model, which eliminates the nonlinearity in GCN and simplifies the propagation of the embedded representation of users or products to alleviate the over-smoothing problem, thereby improving the recommendation accuracy.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A recommendation system method based on the NSGC-GRU integrated model includes the following steps:
[0007] S1: Collect public data sets, process outliers in the data sets, and perform normalization. For each data set, randomly divide it into training, validation, and test sets, and set the ratio of each user to 80:10:10.
[0008] S2: Set the model's hyperparameters, such as the maximum number of training runs, learning rate, batch size, L2 regularization coefficient, etc.
[0009] S3: Based on the data set, a bipartite graph of the relationship between users and products is constructed to generate the adjacency matrix A of the corresponding graph, and then put it into N-SGC training;
[0010] S4: Put the feature fusion results obtained in S3 into the GRU model for training;
[0011] S5: Use the softmax linear model to predict the results obtained in S4;
[0012] S6: Using the predicted data obtained in S5, the recall rate and normalized discounted cumulative gain (NDCG) are calculated to measure the model performance.
[0013] As a preferred technical solution of the present invention: in S2, the hyperparameters of the model are set, the embedding size of all models is fixed to 64, and the embedding parameters are initialized using the Xavier method, the N-SGC model is optimized using Adam, and the learning rate is 0.001 and the batch size is 1024, and the L2 regularization coefficient λ is in {1e -6 ,1e -5 ,…,1e -2}.
[0014] As a preferred technical solution of the present invention: the adjacency matrix A obtained in S3 needs to be symmetric normalized before training:
[0015] A′=A+I
[0016]
[0017] Where D represents the degree matrix and I represents the identity matrix; then it is put into the NSGC model for training; NSGC is a three-layer SGC model, which accumulates the normalized adjacency matrix A, weight θ and feature matrix X, puts them into the model training, and then fuses the three feature matrices obtained, and finally outputs them using the fully connected layer.
[0018] As a preferred technical solution of the present invention: S4 converts the feature fusion result obtained in S3 into time series data, and then uses it as the input of the GRU model and puts it into the GRU model for training;
[0019]
[0020] where h t is the output at the current time t, h t-1 is the output at time t-1, is the candidate state at time t, and the output at the current time t is obtained by weighted summation.
[0021] As a preferred technical solution of the present invention: the softmax formula in S5 is:
[0022]
[0023] Where Z is the output of the previous layer, the input of softmax, the dimension is C, y i is the probability that the predicted object belongs to the Cth category. As a preferred technical solution of the present invention: the calculation formula of the recall rate recall in S6 is:
[0024] recall=TP / (TP+FN)
[0025] Among them, TP is the number of instances correctly classified as positive examples, that is, the number of instances that are actually positive examples and classified as positive examples by the classifier; FN is the number of instances incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but classified as negative examples by the classifier.
[0026] As a preferred technical solution of the present invention: the calculation formula of the normalized cumulative loss gain NDCG in S6 is:
[0027]
[0028] Among them, rel i Represents the relevance of position i, REL p It represents the p lists with the highest output relevance. The results are sorted in descending order of relevance, and the set consisting of the first p results is taken, that is, the results are sorted in the best way.
[0029] The recommendation system method based on the NSGC-GRU integrated model described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:
[0030] The method of the present invention combines neighbor aggregation and time series prediction to recommend to users. GCN can well aggregate the neighbor information of nodes, and then perform collaborative filtering to obtain preliminary recommendation predictions, while GRU can play a good role in time series prediction. The combination of the two can improve the accuracy of recommendation predictions, thereby providing users with more accurate personalized recommendations. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of the method of the present invention;
[0032] Figure 2 Schematic diagram of the comparison of the recall of the LightGCN model of the present invention and the existing method. DETAILED DESCRIPTION
[0033] The present invention is further explained below in detail with reference to the accompanying drawings so that those skilled in the art can have a deeper understanding of the present invention and be able to implement it. However, the following reference examples are only used to explain the present invention and are not intended to limit the present invention.
[0034] like Figure 1 As shown in FIG. 1 , a recommendation system method based on the NSGC-GRU integrated model proposed in the present invention comprises the following steps: S1: by collecting public data from Amazon online, firstly, outlier processing is performed, and users and products with less interactions are removed to prevent the sparse data from having a negative impact on the prediction of the model, and then normalization is performed to obtain a preprocessed data set, and then the user embeddings are extracted. u and product embedding i ,For all datasets, a 10-core setting is used, i.e., users and items with at least 10 interactions are retained.,Finally, for each dataset, it is randomly divided into training, validation, and test sets with a ratio of 80:10:10 for each user;
[0035] S2: Set the model's hyperparameters, such as the maximum number of training runs, learning rate, batch size, L2 regularization coefficient, etc.
[0036] S3: Based on the data set, a bipartite graph of the relationship between users and products is constructed to generate the adjacency matrix A of the corresponding graph, and then put it into N-SGC training;
[0037] S4: Put the feature fusion results obtained in S3 into the GRU model for training;
[0038] S5: Use the softmax linear model to predict the results obtained in S4;
[0039] S6: Using the predicted data obtained in S5, the recall rate and normalized discounted cumulative gain (NDCG) are calculated to measure the model performance.
[0040] In S2, the hyperparameters of the model are set. Given that the embedding size of all models is fixed to 64, the embedding parameters are initialized using the Xavier method, and the N-SGC model is optimized using Adam, with a learning rate of 0.001 and a batch size of 1024. The L2 regularization coefficient λ is in {1e -6 ,1e -5 ,…,1e -2}, in most cases, the optimal value is 1e -4 .
[0041] The adjacency matrix A obtained in S3 needs to be symmetric normalized before training:
[0042] A′=A+I
[0043]
[0044] Where D represents the degree matrix and I represents the identity matrix. Then the model is put into NSGC for training. NSGC is a three-layer SGC model. This can aggregate more neighbor information so that a node can receive information from almost all other nodes, so that the model can achieve the effect of high-order GCN. The normalized adjacency matrix A, weight θ and feature matrix X are accumulated and put into the model for training. After that, the three feature matrices are fused and finally output using the fully connected layer.
[0045] The formula is:
[0046]
[0047] S4 converts the feature fusion result obtained in S3 into time series data, and then uses it as the input of the GRU model and puts it into the GRU model for training;
[0048]
[0049] where h t is the output at the current time t, h t-1 is the output at time t-1, is the candidate state at time t, and the output at the current time t is obtained by weighted summation.
[0050] The softmax formula in S5 is:
[0051]
[0052] Among them, Z is the output of the previous layer, the input of softmax, the dimension is C, y i is the probability that the predicted object belongs to the Cth class. The calculation formula of recall rate recall in S6 is:
[0053] recall=TP / (TP+FN)
[0054] Among them, TP is the number of instances correctly classified as positive examples, that is, the number of instances that are actually positive examples and classified as positive examples by the classifier; FN is the number of instances incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but classified as negative examples by the classifier.
[0055] The calculation formula of normalized discounted cumulative gain NDCG in S6 is:
[0056]
[0057] Among them, rel i Represents the relevance of position i, REL p It represents the p lists with the highest output relevance. The results are sorted in descending order of relevance, and the set consisting of the first p results is taken, that is, the results are sorted in the best way.
[0058] The present invention proposes a recommendation system method based on the NSGC-GRU integrated model, such as Figure 2 As shown in Figure 2, the experimental results show that the proposed method is more effective than other GCN-based methods in recommendation tasks. For example, on this dataset, in terms of the evaluation indicator recall, the proposed method has a 1.87% performance improvement over the existing LightGCN model.
[0059] The specific implementation scheme described above further describes in detail the purpose, technical scheme and beneficial effects of the present invention. It should be understood that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes and modifications made by any technician in the field without departing from the concept and principle of the present invention should fall within the scope of protection of the present invention.
Claims
1. A recommendation system method based on NSGC-GRU integrated model, characterized in that: The following steps are involved: S1: Collect public data sets, process outliers in the data sets, and perform normalization. For each data set, randomly divide it into training, validation, and test sets, and set the ratio of each user to 80:10:
10. S2: Set the model's hyperparameters, including the maximum number of training runs, learning rate, batch size, and L2 regularization coefficient; S3: Based on the data set, a bipartite graph of the relationship between users and products is constructed to generate the adjacency matrix A of the corresponding graph, and then put it into N-SGC training; S4: Put the feature fusion results obtained in S3 into the GRU model for training; S5: Use the softmax linear model to predict the results obtained in S4; S6: Using the predicted data obtained in S5, the recall rate and the normalized discounted cumulative gain (NDCG) are calculated to measure the model performance. The adjacency matrix A obtained in S3 needs to be symmetric normalized before training: A′=A+I Where D represents the degree matrix and I represents the identity matrix. Then put it into the NSGC model for training. NSGC is a three-layer SGC model. The normalized adjacency matrix A, weight θ and feature matrix X are accumulated and put into the model for training. Then the three feature matrices are fused and finally output using the fully connected layer. S4 converts the feature fusion result obtained in S3 into time series data, and then uses it as the input of the GRU model and puts it into the GRU model for training; where h t is the output at the current time t, h t-1 is the output at time t-1, is the candidate state at time t, and the output at the current time t is obtained by weighted summation.
2. The recommendation system method based on the NSGC-GRU integrated model according to claim 1, characterized in that: In S2, the hyperparameters of the model are set. The embedding size of all models is fixed to 64, and the embedding parameters are initialized using the Xavier method. Adam is used to optimize the N-SGC model, and the learning rate is used. 0.001 and the batch size is 1024, the L2 regularization coefficient λ is in {1e -6 ,1e -5 ,…,1e -2 }.
3. The recommendation system method based on the NSGC-GRU integrated model according to claim 1, characterized in that: The softmax formula in S5 is: Among them, Z is the output of the previous layer, the input of softmax, the dimension is C, y i is the probability that the predicted object belongs to the Cth class.
4. The recommendation system method based on the NSGC-GRU integrated model according to claim 1, characterized in that: The calculation formula of the recall rate recall in S6 is: recall=TP / (TP+FN) Among them, TP is the number of instances correctly classified as positive examples, that is, the number of instances that are actually positive examples and classified as positive examples by the classifier; FN is the number of instances incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but classified as negative examples by the classifier.
5. The recommendation system method based on the NSGC-GRU integrated model according to claim 1, characterized in that: The calculation formula of the normalized discounted cumulative gain NDCG in S6 is: Among them, rel i Represents the relevance of position i, REL p It represents the p lists with the highest output relevance. The results are sorted in descending order of relevance, and the set consisting of the first p results is taken, that is, the results are sorted in the best way.