A scoring sequence recommendation method based on a lightweight graph convolution self-attention model

Abstract

The application discloses a scoring sequence recommendation method based on a lightweight graph convolution self-attention model, which comprises the following steps: 1, data acquisition and preprocessing; 2, dividing the data set; 3, constructing and training the lightweight graph convolution self-attention model; 4, predicting through the trained lightweight graph convolution self-attention model. The application generates an interaction graph of users and projects through the interaction of users and projects, and then generates an interaction matrix, introduces the scores of users to projects in the interaction matrix, captures the spatial features of data through GCN, and then introduces the obtained features into position coding through self-attention to capture the time features of data, which considers the score weight of users to projects and accurately predicts the projects that users may interact from the two dimensions of time and space. Meanwhile, the dependence between the spatial and time features of data is considered, and the self-attention of position coding can capture the long-term interest of users, so that the accuracy of the recommendation model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of recommendation systems, and particularly to a scoring sequence recommendation method based on a lightweight graph convolution self-attention model. BACKGROUND

[0002] With the rapid development of Internet technology, excessive fragmented information has flooded people's lives, making people dazzled. The recommendation system is a technology that has emerged to alleviate the problem of information overload on the network. It can help users quickly find information and goods of interest, thereby improving user experience. The core technology of the recommendation system is to use a large amount of user historical behavior data and item information to predict user interests and behaviors through machine learning algorithms, thereby providing personalized recommendations to users. With the increasing size of data and the continuous improvement of algorithms, the recommendation system has become an indispensable part of the Internet industry, and it is widely used in e-commerce, social networks, music and film services and other fields.

[0003] Most of the recommendation models used in the prior art can only obtain the features of one of the time or space dimensions, and cannot capture the spatio-temporal features of user and item data at the same time. For example, the neural graph collaborative filtering (NGCF) and the simplifying and powering graph convolution network for recommendation (LightGCN) models use the graph convolutional neural network (GCN) to capture the spatial features of the data, but cannot handle time correlation. Another type of sequence recommendation model such as self-attentive sequential recommendation (SASRec) uses Markov chain (MC), recurrent neural network (RNN) and Transformer methods to model the time series, but cannot capture the spatial information between data and requires a large amount of data for training. Most of the current models cannot accurately model the spatial and temporal features of the data, and these methods only simply consider the interaction between users and items without considering feedback information such as user ratings on items, making it difficult to accurately capture user interests.

[0004] At present, some designers also combine the recommendation model capturing the spatial features and the recommendation model capturing the temporal features to form a recommendation model that can capture the spatial features and the temporal features. However, these models often capture the spatial features and the temporal features by parallel connection, and then fuse the feature representations. These methods do not consider the dependency between the spatial and temporal features of the data, and only consider the interaction relationship between the user and the item, ignoring the feedback information such as the rating, and cannot accurately obtain the complete features of the data.

[0005] There is an urgent need for a new recommendation system model to solve the above problems. SUMMARY

[0006] The present application provides a rating sequence recommendation method based on a lightweight graph convolution self-attention model, which solves the problem of inaccurate recommendation in the prior art.

[0007] The technical solution of the present application is as follows: a rating sequence recommendation method based on a lightweight graph convolution self-attention model, the specific steps are as follows:

[0008] Step 1: Obtain the historical interaction data of users and items, and preprocess, including sparse value elimination, and generating interaction sequences by time sorting;

[0009] Step 2: Divide the data set; from sequence to input and output sequence pair; divide the data set into training set and test set;

[0010] Step 3: Neural network training based on lightweight graph convolution self-attention model, construct the interaction matrix R of users and items, and introduce the rating information of users to items rating, wherein M and N represent the number of users and items respectively, if the user u and the item i interact, then each item R M×N The value of each item R ui is rating, otherwise 0, and the interaction matrix of users and items is represented as:

[0011]

[0012] Step 4: Neural network prediction based on lightweight graph convolution self-attention model;

[0013] Further, the step 1 is specifically: (1) obtaining the interaction of users to items, interaction time and interaction rating; (2) sparse value elimination, directly deleting data with too few user and item interaction times; (3) sorting the user and item interaction by time to generate interaction sequences.

[0014] Further, the step 2 is specifically: (1) setting the maximum length N, controlling the length of the user and the project interaction sequence to N, when the sequence length is greater than N, only considering the last N interactions of the user, when the sequence length is less than N, zero padding to the left of the user and the project interaction sequence until the sequence length is N; (2) dividing the data set, taking the first N-1 items of the user and the project interaction sequence as the training set, and taking the Nth item as the test set.

[0015] Further, the step 3 is specifically: (1) building a model based on light graph convolution self-attention, and using the method of Xavier initialization to initialize the parameters of the model; (2) inputting the training set into the model for calculation, introducing the score into the user and the project interaction matrix in the GCN part, using the binary cross entropy loss function, and using Adam as the optimization algorithm to obtain the prediction result when the training converges.

[0016] Further, the step 4 is specifically: (1) using the trained prediction model to predict the user's possible interaction projects on the test set; (2) comparing all the prediction results with the true values, using Recall and NDCG as the evaluation indicators of the model, and obtaining the performance evaluation result of the model; (3) the final prediction model is used for the recommendation of the actual recommendation system.

[0017] The beneficial effects of the present application are:

[0018] The score sequence recommendation method based on the light graph convolution self-attention model disclosed in the present application generates a user and project interaction graph from the user and project interaction, and then generates an interaction matrix, introduces the user's score on the project into the interaction matrix, captures the spatial features of the data through GCN, and then introduces the obtained features through self-attention to capture the time features of the data, which considers the score weight of the user on the project, and accurately predicts the projects that the user may interact from the time and space dimensions. At the same time, the dependence between the spatial and temporal features of the data is considered, and the self-attention of the position coding is added to capture the long-term interest of the user, thereby improving the accuracy of the recommendation model. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0020] Figure 1 is a light graph convolution self-attention framework schematic diagram according to an embodiment of the present application;

[0021] Figure 2 is a whole flow schematic diagram according to an embodiment of the present application;

[0022] Figure 3 is a model evaluation result diagram on three real world data sets according to different dimensions of embodiments of the application;

[0023] Figure 4 is a model evaluation result diagram on three real world data sets according to different numbers of self-attention heads of embodiments of the application. DETAILED DESCRIPTION

[0024] The application will be further described below with reference to the accompanying drawings of the embodiments of the application.

[0025] As shown in Figure 1 and 2 , the application discloses a scoring sequence recommendation method based on a lightweight graph convolution self-attention model, and the specific steps are as follows:

[0026] Step 1: Obtain the historical interaction data of users and items and pre-process, including sparse value elimination, time sorting to generate interaction sequences;

[0027] Step 2: Divide the data set; from sequence to input and output sequence pair; divide the data set into training set and test set;

[0028] Step 3: Neural network training based on lightweight graph convolution self-attention model, when constructing the interaction matrix R of users and items M×N , introduce the rating information of users to items, where M and N represent the number of users and items respectively, if user u and item i interact, then each item R ui The value is rating, otherwise 0, and the interaction matrix of users and items is represented as:

[0029]

[0030] It can be understood that the rating information in the embodiment specifically refers to the score of the user to the item.

[0031] Step 4: Neural network prediction based on lightweight graph convolution self-attention model;

[0032] Further, the step 1 is specifically: (1) obtaining the interaction of users to items, interaction time and interaction score; (2) sparse value elimination, directly deleting data with too few user and item interaction times; (3) sorting the user and item interaction by time to generate interaction sequences.

[0033] Further, the step 2 is specifically: (1) setting the maximum length N, controlling the length of the user and the project interaction sequence to N, when the sequence length is greater than N, only considering the user's last N interactions, when the sequence length is less than N, zero padding the left side of the user and the project interaction sequence until the sequence length is N; (2) dividing the data set, taking the first N-1 items of the user and the project interaction sequence as the training set, and the Nth item as the test set.

[0034] Further, the step 3 is specifically: (1) building a model based on light-weight graph convolution self-attention, and using the method of Xavier initialization to initialize the parameters of the model; (2) inputting the training set into the model for calculation, introducing the score into the user and the project interaction matrix in the GCN part,

[0035] Specifically, the light-weight graph convolution operation is defined as:

[0036]

[0037]

[0038] wherein and respectively represent the refined embedding of user u and project i after k layers of propagation, N u represents the set of projects interacted by user u, N i represents the set of users interacted with project i. The obtained embedding is calculated by self-attention after adding position encoding according to the following formula:

[0039]

[0040] Finally, the binary cross entropy loss function is adopted, Adam is used as the optimization algorithm, and the prediction result when the training converges is obtained.

[0041] Further, the step 4 is specifically: (1) using the trained prediction model to predict the user's possible interaction projects on the test set; (2) comparing all the prediction results with the true values, using Recall and NDCG as the evaluation indicators of the model, and obtaining the performance evaluation result of the model; (3) the final prediction model is used for the recommendation of the actual recommendation system.

[0042] To verify the effect of the present application, the recommended performance of the model is verified on three real data sets. These data sets differ significantly in domain, size and sparsity. Amazon-game is part of Amazon's public data set, which has high sparsity features. The Book-crossing data set is collected from the Book-Crossing community for four weeks through a crawler. Movielens is a widely used movie recommendation data set. The final model results and their ablation studies are shown in Table 1, which shows the impact of each part of the model on the model. It can be seen that the recommended performance of the complete model is the best, the graph convolution part has the greatest impact on the model performance, and the self-attention module is second. The non-linear activation and score parts have less impact on the overall model, but can still achieve the effect of improving the performance of the model. Figure 3 and Figure 4 The influence of different dimensions and the number of self-attention heads on the performance of the model is shown, and it can be seen that the method in the present application achieves relatively accurate recommended performance.

[0043] Table 1: Model results and their ablation studies

[0044]

[0045]

[0046] In summary, the above is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A rating sequence recommendation method based on a lightweight graph convolutional self-attention model, characterized by: The following steps are involved: Step 1: Obtain historical interaction data between users and projects and perform preprocessing, including removing sparse values ​​and generating interaction sequences by time sorting; The historical data obtained includes whether the user has interacted with the project, the interaction time and the interaction score; Step 2: Divide the dataset into training and test sets. Step 3: Based on the neural network training of the lightweight graph convolution self-attention model, the interaction matrix R∈R between users and items is constructed. M×N When the user's rating information for the project is introduced, M and N represent the number of users and projects respectively. If user u interacts with project i, each entry R ui The value of is rating, otherwise it is 0. The interaction matrix between users and items is expressed as: Specifically include: Step 3.1: Build a lightweight graph convolutional self-attention model and use the Xavier initialization method to initialize the parameters of the lightweight graph convolutional self-attention model; The graph convolution network part of the lightweight graph convolution self-attention model introduces the rating into the interaction matrix between users and items. The lightweight graph convolution operation is defined as: in and They represent the refined embeddings of user u and item i after k-layer propagation, N u represents the set of items interacted by user u, N i represents the set of users who interacted with item i. The resulting embedding is then self-attentioned using the following formula after adding positional encoding: Finally, the binary cross entropy loss function and Adam are used as the optimization algorithm to obtain the prediction results when training to convergence; Step 3.2: Input the training set into the lightweight graph convolution self-attention model after parameter initialization for calculation; Step 4: Neural network prediction based on lightweight graph convolutional self-attention model.

2. The method for recommending a scoring sequence based on a lightweight graph convolutional self-attention model according to claim 1, wherein: The method for preprocessing historical data in step 1 is: first, sparse value elimination is performed to directly delete data with too few interactions between users and projects; then, the interactions between users and projects are sorted by time to generate an interaction sequence.

3. The method for recommending a scoring sequence based on a lightweight graph convolutional self-attention model according to claim 2, wherein: In step 1, the method for preprocessing historical data also includes: setting a maximum length N, controlling the length of the interaction sequence between the user and the project to N, when the sequence length is greater than N, only considering the user's most recent N interactions, and when the sequence length is less than N, padding the left side of the interaction sequence between the user and the project with zeros until the sequence length reaches N.

4. The method for recommending a scoring sequence based on a lightweight graph convolutional self-attention model according to claim 3, wherein: In step 2, the method of dividing the data set is: the first N-1 items of the interaction sequence between the user and the item are used as the training set, and the Nth item is used as the test set.

5. The method for rating sequence recommendation based on a lightweight graph convolutional self-attention model according to claim 1, characterized in that: The step 4 is specifically as follows: Step 4.1: Use the trained lightweight graph convolutional self-attention model on the test set to predict the items that the user may interact with; Step 4.2: Compare all prediction results with the true values, use Recall and NDCG as the evaluation indicators of the model, and obtain the performance evaluation results of the model; Step 4.3: The final lightweight graph convolutional self-attention model is used for recommendations in actual recommendation systems.

Images