A neural collaborative filtering recommendation method combining attention mechanism and autoencoder

Abstract

The present invention belongs to the technical field of recommendation algorithms, and specifically relates to a neural collaborative filtering recommendation method combining an attention mechanism with an autoencoder, comprising: collecting user and product information data, and constructing a user-item interaction matrix; based on the user-item interaction matrix, obtaining a user feature vector and an item feature vector; inputting the user feature vector and the item feature vector into a deep matrix decomposition model based on an attention mechanism and an autoencoder, respectively, to obtain a first prediction vector; inputting the user feature vector and the item feature vector into a multilayer perceptron model based on an attention mechanism and an autoencoder, respectively, to obtain a second prediction vector; combining the deep matrix decomposition model and the multilayer perceptron model based on preset model weights, and fusing the first prediction vector and the second prediction vector to obtain a joint prediction score; based on the obtained joint prediction score, generating a product recommendation list to recommend items to users.

Description

Technical Field

[0001] The present invention belongs to the technical field of recommendation algorithms, and in particular relates to a neural collaborative filtering recommendation method combining an attention mechanism with an autoencoder. Background Art

[0002] With the development of science and technology and the improvement of people's living standards, the scale and coverage of the Internet are getting larger and larger, and the amount of information data generated is exploding. The excessive information requires users to spend a lot of time filtering and integrating information to select effective information. In order to improve people's effective use of information, recommendation systems have emerged. They can not only meet users' information needs, but also, unlike search algorithms, recommendation systems can actively make personalized recommendations based on different users. The recommendation algorithm is the key to determining the accuracy of the recommendation system.

[0003] As a type of collaborative filtering, the recommendation algorithm based on matrix decomposition has been widely studied. With the development of deep learning technology, more and more deep learning technologies are being used to learn features of vectors or data information, thereby mining implicit feedback information of the data. For example, the attention mechanism can be used to assign weights to targets and perform representation learning in a focused manner. Therefore, the improvement of recommendation algorithms based on traditional matrix decomposition or collaborative filtering combined with deep learning technology has been widely studied and applied.

[0004] When modeling in existing collaborative filtering models, the user feature vector and the item feature user vector are often combined first, and then input into the attention layer for calculation. However, the filtering effect of this method often has certain limitations, which in turn affects the accuracy of the recommendation system. Summary of the invention

[0005] The present invention provides a neural collaborative filtering recommendation method combining an attention mechanism with an autoencoder. The method constructs a user-item interaction matrix, and based on the user feature vector and the item feature vector obtained through the user-item interaction matrix, first inputs them into a deep matrix decomposition model based on an attention mechanism and an autoencoder for learning to obtain a first prediction vector, so as to obtain more implicit feedback information, thereby being more representative of the user and item features, and then inputs them into a multi-layer perceptron model based on an attention mechanism and an autoencoder to obtain a second prediction vector, and the first prediction vector and the second prediction vector are jointly calculated to make the recommendation result more accurate.

[0006] A neural collaborative filtering recommendation method combining an attention mechanism and an autoencoder comprises the following steps:

[0007] Collect user and product information data and construct a user-item interaction matrix;

[0008] Based on the user-item interaction matrix, obtain the user feature vector and the item feature vector;

[0009] Input the user feature vector and the item feature vector into the deep matrix decomposition model based on the attention mechanism and the autoencoder respectively to obtain the first prediction vector;

[0010] The user feature vector and the item feature vector are respectively input into the multi-layer perceptron model based on the attention mechanism and the autoencoder to obtain the second prediction vector;

[0011] Based on the preset model weights, the deep matrix decomposition model and the multi-layer perceptron model are combined, and the first prediction vector and the second prediction vector are fused to obtain a joint prediction score;

[0012] Based on the obtained joint prediction score, a product recommendation list is generated to recommend items to users.

[0013] By constructing a user-item interaction matrix, and based on the user feature vector and item feature vector obtained through the user-item interaction matrix, first input it into a deep matrix decomposition model based on an attention mechanism and an autoencoder for learning to obtain a first prediction vector, so as to obtain more implicit feedback information, and thus be more representative of user and item features, and then input it into a multi-layer perceptron model based on an attention mechanism and an autoencoder to obtain a second prediction vector, and then jointly calculate the first prediction vector and the second prediction vector to make the recommendation results more accurate.

[0014] Furthermore, the process of collecting user and product information data and constructing a user-item interaction matrix specifically includes the following steps:

[0015] Collect information data of user ID, item ID, and user's rating of items respectively, and pre-process the data;

[0016] Based on the preprocessed data, the interaction between the user and the item is obtained, and "1" is used to indicate that there is interaction between the user and the item, and "0" is used to indicate that there is no interaction between the user and the item;

[0017] The user-item interaction matrix Y is constructed by using rows to represent user IDs, columns to represent item IDs, and the values ​​in the matrix to represent the user's interaction with the items.

[0018] Furthermore, based on the user-item interaction matrix, the process of obtaining the user feature vector and the item feature vector specifically includes:

[0019] According to the rows in the user-item interaction matrix, the user feature vectors corresponding to the user ID are obtained, that is, Represents the user feature vector of the Uth row in the user-item interaction matrix Y;

[0020] According to the columns in the user-item interaction matrix, the item feature vectors corresponding to the item ID are obtained respectively, that is, represents the first i The user feature vector of the column.

[0021] Furthermore, the user feature vector and the item feature vector are input into the deep matrix decomposition model based on the attention mechanism and the autoencoder, and the process of obtaining the first prediction vector specifically includes the following steps:

[0022] The user feature vector is input into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the user's first decoding vector; and the obtained user's first decoding vector and the user feature vector are jointly input into the multi-layer perceptron for learning to obtain the user's potential feature vector;

[0023] The item feature vector is input into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the first decoding vector of the item; and the obtained first decoding vector of the item and the user feature vector are jointly input into the multi-layer perceptron for learning to obtain the potential feature vector of the item;

[0024] The dot product calculates the similarity between the user's potential feature vector and the item's potential feature vector, and combines the corresponding weight matrix and activation function to obtain the first prediction vector.

[0025] Furthermore, the calculation expression of the user's first decoding vector or the item's first decoding vector is:

[0026] ;

[0027] In the formula, Decode the vector for the user first; is the input vector of the input layer in the deep matrix decomposition model, that is, , is the transposed matrix of the feedforward attention layer weight matrix, is the user-item interaction matrix The user ID is The transposed vector of to the user feature vector; is the attention ratio of the attention layer, i.e. , is the Softmax activation function, is the transposed matrix of the weight matrix of the BP neural network, is the bias vector of the BP neural network;

[0028] User latent feature vector Or the item latent feature vector The calculation expression is:

[0029] ;

[0030] In the formula, is the user's potential feature vector; is the item’s potential feature vector; is the activation of the Xth layer of the BP neural network; is the ReLU activation function; is the transposed matrix of the X-th layer weight matrix of the BP neural network; is the bias vector of the Xth layer of the BP neural network;

[0031] Similarity between user latent feature vector and item latent feature vector The calculation expression is:

[0032] ;

[0033] The first prediction vector The calculation expression is:

[0034] ;

[0035] In the formula, Similarity scores for users and items in deep matrix factorization models; is the Sigmod activation function; is the weight matrix of the output layer in the deep matrix factorization model.

[0036] Furthermore, the user feature vector and the item feature vector are respectively input into the multi-layer perceptron model based on the attention mechanism and the autoencoder, and the process of obtaining the second prediction vector specifically includes the following steps:

[0037] The parameter matrix of the linear embedding layer in the multi-layer perceptron model is used to transform the user feature vector into a user linear feature vector and the item feature vector into an item linear feature vector;

[0038] The user linear feature vector and the item linear feature vector are respectively input into the attention layer of the multi-layer perceptron model for encoding and decoding, and the user second decoding vector and the item second decoding vector are respectively obtained;

[0039] The user's second decoding vector and the item's second decoding vector are combined and used as the input vector of the multi-layer perceptron. The second prediction vector is obtained by combining the corresponding weight matrix and activation function.

[0040] Furthermore, the user linear feature vector The calculation expression is:

[0041] ;

[0042] In the formula, is the transposed matrix of the parameter matrix of the relinear embedding layer of the multi-layer perceptron model;

[0043] Item linear feature vector The calculation expression is:

[0044] ;

[0045] In the formula, The transposed matrix of the parameter matrix of the relinear embedding layer of the multi-layer perceptron model;

[0046] User second decoding vector The calculation expression is:

[0047] ;

[0048] In the formula, A second decoding vector for the user; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ;

[0049] Item second decoding vector The calculation expression is:

[0050] ;

[0051] In the formula, is the second decoding vector of the item; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ;

[0052] The input vector of the multilayer perceptron for:

[0053] ;

[0054] In the formula, is the input vector of the multilayer perceptron;

[0055] The second prediction vector The calculation expression is:

[0056] ;

[0057] In the formula, Similarity scores for users and items in the multi-layer perceptron model; is the Sigmod activation function; is the weight matrix of the input layer in the multi-layer perceptron model; is the output vector of the Yth layer of the deep network, that is , is the transposed matrix of the Y-th layer weight matrix of the BP neural network; is the bias vector of the Yth layer of the BP neural network.

[0058] Furthermore, the process of combining the deep matrix decomposition model and the multi-layer perceptron model based on the preset model weights and fusing the first prediction vector and the second prediction vector to obtain a joint prediction score specifically includes the following steps:

[0059] The deep matrix decomposition model and the multi-layer perceptron model are combined to build a joint model for training, and the model weights of both are set to 1 / 2;

[0060] The first prediction vector output by the deep matrix decomposition model and the second prediction vector output by the multi-layer perceptron model are fused, and the corresponding weight matrix and activation function are combined to obtain a joint prediction score;

[0061] Joint prediction scoring The calculation expression is:

[0062] ;

[0063] In the formula, represents the joint prediction score, Represents the weight matrix of the output layer of the joint model.

[0064] Furthermore, the process of training the joint model specifically includes the following steps:

[0065] Based on the interaction between users and items, and assuming that the interaction between users and items follows Bernoulli distribution, a likelihood function is defined to predict the interaction probability between users and items.

[0066] Among them, the likelihood function Defined as:

[0067] ;

[0068] In the formula, For actual rating; Represents all the interactions between users and items that can be observed in the sampled data. Indicates the interactions between users and items that cannot be observed in the sampled data, that is, the number of negative samples;

[0069] The binary cross entropy loss function is used to calculate the loss value of the joint model;

[0070] Among them, the calculation expression of the binary cross entropy loss function is:

[0071] ;

[0072] The HR index and NDCG index are used to evaluate the performance of the joint model;

[0073] Among them, the calculation expression of HR index is:

[0074] ;

[0075] In the formula, Indicates Whether the item clicked by the user is in the recommendation list, that is, 1 is used to indicate if it is in the list, and 0 is used to indicate if it is not in the list; Indicates the total number of users;

[0076] The calculation expression of NDCG index is:

[0077] ;

[0078] In the formula, ;

[0079] In the formula, ; ; Indicates the item score; Indicates item score Sort from largest to smallest and take the first A collection of results.

[0080] By constructing a binary cross entropy loss function, the joint model is trained iteratively to obtain the optimal model training result; at the same time, HR and NDCG evaluation indicators are used to facilitate the evaluation of the accuracy of the model.

[0081] The beneficial effects of the present invention are:

[0082] The present invention constructs a user-item interaction matrix, and based on the user feature vector and item feature vector obtained through the user-item interaction matrix, first inputs it into a deep matrix decomposition model based on an attention mechanism and an autoencoder for learning to obtain a first prediction vector, so as to obtain more implicit feedback information, and thus be more representative of user and item features, and then inputs it into a multi-layer perceptron model based on an attention mechanism and an autoencoder to obtain a second prediction vector, and jointly calculates the first prediction vector and the second prediction vector to make the recommendation result more accurate. The joint model is trained iteratively by constructing a binary cross entropy loss function to obtain the optimal model training result; at the same time, the HR and NDCG evaluation indicators are used to facilitate the evaluation of the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 is a flow chart of the present invention;

[0084] Figure 2 It is a schematic diagram of the algorithm of the present invention;

[0085] Figure 3 Schematic diagram of the relationship between the binary cross entropy loss function and the number of iterations;

[0086] Figure 4 Schematic diagram of the change of HR index by negative sampling under different data sets;

[0087] Figure 5 Schematic diagram of the change of NDCG index by negative sampling under different data sets;

[0088] Figure 6 Schematic diagram of the effects of HR and NDCG indicators under different numbers of latent factors on the m1-100k dataset. DETAILED DESCRIPTION

[0089] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0090] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0091] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples, and a person of ordinary skill in the art can understand the specific meanings of the above terms in this application under specific circumstances.

[0092] Example 1

[0093] Figure 1The method shown is a neural collaborative filtering recommendation method combining attention mechanism and autoencoder, which constructs a user-item interaction matrix, and based on the user feature vector and item feature vector obtained through the user-item interaction matrix, first inputs it into a deep matrix decomposition model based on attention mechanism and autoencoder for learning to obtain a first prediction vector, so as to obtain more implicit feedback information, and thus be more representative of user and item features, and then inputs it into a multi-layer perceptron model based on attention mechanism and autoencoder to obtain a second prediction vector, and jointly calculates the first prediction vector and the second prediction vector to make the recommendation result more accurate. Specifically, the following steps are included:

[0094] S1: Collect user and product information data and construct a user-item interaction matrix;

[0095] S11: Collect information data of user ID, item ID, and user's rating of the item respectively, and pre-process the data;

[0096] S12: Based on the preprocessed data, the interaction between the user and the item is obtained, and "1" is used to indicate that there is interaction between the user and the item, and "0" is used to indicate that there is no interaction between the user and the item;

[0097] S13: Construct a user-item interaction matrix Y by using rows to represent user IDs, columns to represent item IDs, and values ​​in the matrix to represent the user's interaction with the item.

[0098] S2: Based on the user-item interaction matrix, obtain the user feature vector and item feature vector;

[0099] S21: Obtain the user feature vector corresponding to the user ID according to the rows in the user-item interaction matrix, that is, represents the first u User feature vector of the row;

[0100] S22: Obtain the item feature vectors corresponding to the item IDs according to the columns in the user-item interaction matrix, that is, represents the first i The user feature vector of the column.

[0101] In this embodiment, the user-item interaction matrix is ​​an M×N matrix.

[0102] S3: Input the user feature vector and the item feature vector into the deep matrix decomposition model based on the attention mechanism and the autoencoder respectively to obtain the first prediction vector;

[0103] S31: Input the user feature vector into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the user's first decoding vector; and input the obtained user's first decoding vector and the user feature vector into a multi-layer perceptron for learning to obtain the user's potential feature vector;

[0104] The calculation expression of the user's first decoding vector is:

[0105] ;

[0106] In the formula, Decode the vector for the user first; is the input vector of the input layer in the deep matrix decomposition model, that is, , is the transposed matrix of the feedforward attention layer weight matrix, is the user-item interaction matrix The user ID is The transposed vector of to the user feature vector; is the attention ratio of the attention layer, i.e. , is the Softmax activation function, is the transposed matrix of the weight matrix of the BP neural network, is the bias vector of the BP neural network;

[0107] Among them, the calculation expression of the user's potential feature vector is:

[0108] ;

[0109] In the formula, is the user's potential feature vector; is the item’s potential feature vector; is the activation of the Xth layer of the BP neural network; is the ReLU activation function; is the transposed matrix of the X-th layer weight matrix of the BP neural network; is the bias vector of the Xth layer of the BP neural network;

[0110] S32: Input the item feature vector into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the first decoding vector of the item; and input the obtained first decoding vector of the item and the user feature vector into a multi-layer perceptron for learning to obtain a potential feature vector of the item;

[0111] The same method as step S1 is used to calculate the potential feature vector of the item .

[0112] S33: dot product is used to calculate the similarity between the user's potential feature vector and the item's potential feature vector, and a first prediction vector is obtained by combining the corresponding weight matrix and activation function.

[0113] Among them, the similarity between the user's latent feature vector and the item's latent feature vector The calculation expression is:

[0114] ;

[0115] Among them, the first prediction vector The calculation expression is:

[0116] ;

[0117] In the formula, Similarity scores for users or items in deep matrix factorization models; is the Sigmod activation function; is the weight matrix of the output layer in the deep matrix factorization model.

[0118] S4: Input the user feature vector and the item feature vector into the multi-layer perceptron model based on the attention mechanism and the autoencoder respectively to obtain the second prediction vector;

[0119] S41: using the parameter matrix of the linear embedding layer in the multi-layer perceptron model to transform the user feature vector into a user linear feature vector and the item feature vector into an item linear feature vector;

[0120] Among them, the user linear feature vector The calculation expression is:

[0121] ;

[0122] In the formula, is the transposed matrix of the parameter matrix of the relinear embedding layer of the multilayer perceptron model, and ;

[0123] Among them, the item linear feature vector The calculation expression is:

[0124] ;

[0125] In the formula, The transposed matrix of the parameter matrix of the relinear embedding layer of the multilayer perceptron model, and ;

[0126] S42: Inputting the user linear feature vector and the item linear feature vector into the attention layer of the multi-layer perceptron model for encoding and decoding respectively, and obtaining the user second decoding vector and the item second decoding vector respectively;

[0127] Among them, the user's second decoding vector The calculation expression is:

[0128] ;

[0129] In the formula, A second decoding vector for the user; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ;

[0130] Among them, the second decoding vector of the item The calculation expression is:

[0131] ;

[0132] In the formula, is the second decoding vector of the item; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ;

[0133] S43: Combine the user's second decoding vector and the item's second decoding vector, use them as input vectors of the multi-layer perceptron, and combine the corresponding weight matrix and activation function to obtain a second prediction vector.

[0134] Among them, the input vector of the multilayer perceptron for:

[0135] ;

[0136] In the formula, is the input vector of the multilayer perceptron;

[0137] Among them, the second prediction vector The calculation expression is:

[0138] ;

[0139] In the formula, Similarity scores for users and items in the multi-layer perceptron model; is the Sigmod activation function; is the weight matrix of the input layer in the multi-layer perceptron model; is the output vector of the Yth layer of the deep network, that is , is the transposed matrix of the Y-th layer weight matrix of the BP neural network; is the bias vector of the Yth layer of the BP neural network.

[0140] S5: combining the deep matrix decomposition model and the multi-layer perceptron model based on the preset model weights, and fusing the first prediction vector and the second prediction vector to obtain a joint prediction score;

[0141] S51: The deep matrix decomposition model and the multi-layer perceptron model are combined to build a joint model for training, and the model weights of both are set to 1 / 2;

[0142] S52: fusing the first prediction vector output by the deep matrix decomposition model and the second prediction vector output by the multi-layer perceptron model, and combining the corresponding weight matrix and activation function to obtain a joint prediction score;

[0143] Among them, the joint prediction score The calculation expression is:

[0144] ;

[0145] In the formula, represents the joint prediction score, Represents the weight matrix of the output layer of the joint model.

[0146] In this embodiment, the process of training the joint model specifically includes the following steps:

[0147] Based on the interaction between users and items, we define an objective function suitable for learning from implicit feedback, namely the binary cross loss function, to accurately predict the interaction probability between users and items. We also restrict the joint prediction score interval to [0,1] by using the Logistic function as the activation function of the output layer of the joint model.

[0148] In this embodiment, it is assumed that the interaction between users and items follows the Bernoulli distribution, and a likelihood function is defined to predict the interaction probability between users and items; that is, the likelihood function Defined as:

[0149] ;

[0150] In the formula, For actual rating; Represents all the interactions between users and items that can be observed in the sampled data. Indicates the interactions between users and items that cannot be observed in the sampled data, that is, the number of negative samples; is the likelihood estimation model parameters;

[0151] Among them, maximizing the logarithm of the likelihood function is equivalent to minimizing the binary cross loss function, that is, using the binary cross entropy loss function to calculate the loss value of the joint model to measure the difference between the probability distribution predicted by the joint model and the true distribution;

[0152] Among them, the calculation expression of the binary cross entropy loss function is:

[0153] ;

[0154] The HR index and NDCG index are used to evaluate the performance of the joint model;

[0155] Among them, the calculation expression of HR index is:

[0156] ;

[0157] ;

[0158] In the formula, Indicates Whether the item clicked by the user is in the recommendation list, that is, 1 is used to indicate if it is in the list, and 0 is used to indicate if it is not in the list; Indicates the total number of users;

[0159] The calculation expression of NDCG index is:

[0160] ;

[0161] In the formula, ; ; Indicates the item score; Indicates item score Sort from largest to smallest and take A collection of results.

[0162] S6: Based on the obtained joint prediction score, a product recommendation list is generated to recommend items to the user.

[0163] Example 2

[0164] like Figure 2 As shown, this embodiment provides a neural collaborative filtering recommendation method combining an attention mechanism and an autoencoder, which specifically includes the following steps when performing model training calculation:

[0165] T1: Collect user and product information data and construct a user-item interaction matrix;

[0166] T11: Obtain the information data sets of user ID, item ID, and user's rating of the item, and preprocess the data sets;

[0167] T12: Use the leave-one-out evaluation method to divide the preprocessed data set into a training data set and a test data set; and based on the training data set, randomly select negative samples for each positive sample to construct a negative sampling data set;

[0168] Among them, the positive sample indicates that there is interaction between the user and the item, that is, the user has rated the item, and it is represented by "1"; the negative sample indicates that there is no interaction between the user and the item, that is, the user has not rated the item, and it is represented by "0".

[0169] T13: Obtain the interaction information between users and items, and merge the negative sampling data to build a linear table;

[0170] T14: Use rows to represent user IDs, columns to represent item IDs, and the values ​​in the matrix to represent the user's interaction with items, to construct the user-item interaction matrix Y;

[0171] In this embodiment, the user-item interaction matrix , that is, it is an M×N matrix, M represents the number of users, N represents the number of items; and the user-item interaction matrix.

[0172] T2: Based on the user-item interaction matrix, obtain the user feature vector and item feature vector;

[0173] T21: Obtain the item feature vectors corresponding to the item IDs according to the columns in the user-item interaction matrix, that is represents the first u User feature vector of the row;

[0174] T22: Obtain the item feature vectors corresponding to the item IDs according to the columns in the user-item interaction matrix, that is represents the first i The item feature vector of the column;

[0175] T3: Construct a deep matrix decomposition model based on the attention mechanism and the autoencoder, i.e., the left DMF model, to obtain the first prediction vector;

[0176] T31: Select the training data set and set the model parameters of the DMF model on the left;

[0177] In this embodiment, the model parameters of the DMF model on the left include: the iteration parameter epochs is set to 10, batch_size is set to 256, userlayers is set to [128, 128, 128] and itemlayers is set to [128, 128, 128], the negative sampling num_neg is set to 4, and the learning rate lr is set to 0.00001.

[0178] T32: Input the user feature vector and item feature vector into the attention layer of the left DMF model for learning, and obtain the user's first decoding vector and the item's first decoding vector respectively;

[0179] Among them, the user's first decoding vector The calculation expression is:

[0180] ;

[0181] ;

[0182] ;

[0183] In the formula, represents the user's first decoding vector; Represents the transpose of a vector or matrix; represents the feedforward attention layer weight matrix, and ; is the input vector of the input layer of the left DMF model, and ; is the attention ratio of the attention layer; is the Softmax activation function, is the weight matrix of the BP neural network, is the bias vector of the BP neural network, and ; represents the first u The user feature vector of the row, and ;

[0184] Item first decode vector The calculation expression is:

[0185] ;

[0186] ;

[0187] ;

[0188] In the formula, Represents the first decoding vector of the item; is the input of the left DMF model input layer, and ; is the attention ratio of the attention layer.

[0189] T33: Input the user's first decoding vector and the item's first decoding vector into the deep network of the DMF model on the left to obtain the user's potential feature vector and the item's potential feature vector respectively;

[0190] Among them, the user's first decoding vector is input into the deep network of the DMF model on the left to learn the user's potential feature vector The process includes:

[0191] ;

[0192] ;

[0193] ;

[0194]

[0195] ;

[0196] In the formula, represents the user's potential feature vector, i.e., the output vector of the Xth layer of the deep network, and ; Represents the output vector of the Xth layer of the deep network; Represents the weight matrix of the Xth layer of the deep network; Represents the bias vector of the Xth layer of the deep network; is the ReLU activation function; X is the number of layers in the deep network; is defined as the input to the left-hand DMF model, and ;

[0197] The first decoding vector of the item is input into the deep network of the DMF model on the left to learn the potential feature vector of the item. The process includes:

[0198] ;

[0199] ;

[0200] ;

[0201]

[0202] ;

[0203] In the formula, represents the potential feature vector of the item, i.e., the output vector of the Xth layer of the deep network, and ; Represents the output vector of the Xth layer of the deep network; is defined as the input to the left-hand DMF model, and .

[0204] T34: dot product is used to calculate the similarity between the user's potential feature vector and the item's potential feature vector, and the first prediction vector is calculated;

[0205] Similarity between user latent feature vector and item latent feature vector The calculation expression is:

[0206] ;

[0207] In the formula, Represents the similarity between the user's latent feature vector and the item's latent feature vector;

[0208] The first prediction vector The calculation expression is:

[0209] ;

[0210] In the formula, Similarity scores for users or items in deep matrix factorization models; is the Sigmod activation function; is the weight matrix of the output layer in the deep matrix factorization model.

[0211] T35: Pre-train the DMF model on the left and save the optimal model weights;

[0212] T4: Construct a multi-layer perceptron model based on the attention mechanism and autoencoder, that is, the MLP2 model on the right, to obtain the second prediction vector;

[0213] T41: Select the training data set and set the model parameters of the MLP2 model on the right;

[0214] In this embodiment, the model parameters of the MLP2 model on the right include: the iteration parameter epochs is set to 10, the batch_size is set to 256, the layers are set to [1024, 512, 256, 128], the negative sampling num_neg is set to 4, and the learning rate lr is set to 0.00001.

[0215] T42: Input the user feature vector and item feature vector into the attention layer of the MLP2 model on the right for learning, and obtain the user second decoding vector and item second decoding vector respectively;

[0216] The process of inputting the user feature vector into the attention layer of the MLP2 model on the right side for learning to obtain the user's second decoding vector includes:

[0217] ;

[0218] ;

[0219] ;

[0220] ;

[0221] ;

[0222] ;

[0223] In the formula, express; Represents the user linear vector of the MLP2 model on the right; , are the input vectors of the user linear vector and item linear vector in the input layer of the MLP2 model on the right; , They represent the user's second decoding vector and the item's second decoding vector respectively.

[0224] T43: Combine the user's second decoding vector and the item's second decoding vector, input them into a multi-layer perceptron for learning, and obtain a second prediction vector;

[0225] The vector that combines the user's second decoded vector and the item's second decoded vector It is expressed as:

[0226] ;

[0227] In the formula, represents the vector combining the second decoding vector of the user and the second decoding vector of the item, i.e., the input vector of the multilayer perceptron;

[0228] Among them, the second prediction vector is obtained by learning in the multi-layer perceptron The process includes:

[0229] ;

[0230] ;

[0231]

[0232] ;

[0233] ;

[0234] In the formula, represents the second prediction vector; Score the similarity between users and items in the multi-layer perceptron model; is the Sigmod activation function; is the weight matrix of the input layer in the multi-layer perceptron model; is the weight matrix of the Yth layer of the BP neural network; is the bias vector of the Yth layer of the BP neural network.

[0235] T44: Pre-train the MLP2 model on the right and save the optimal model weights;

[0236] T5: Based on the preset weights, the DMF model on the left and the MLP2 model on the right are combined to build a joint ANCF model, and the first prediction vector and the second prediction vector are fused to obtain a joint prediction score;

[0237] T51: Select the training data set and set the model parameters of the joint ANCF model;

[0238] In this embodiment, the model parameters of the joint ANCF model include: the iteration parameter epochs is set to 10, batch_size is set to 256, userlayers is set to [128, 128, 128] and itemlayers is set to [128, 128, 128], layers is set to [1024, 512, 256, 128], negative sampling num_neg is set to 4, and the learning rate lr is set to 0.00001.

[0239] T52: Import the pre-trained DMF model on the left and MLP2 model on the right, and set the model weights to 1 / 2;

[0240] T53: Fusing the first prediction vector and the second prediction vector, and outputting the combined prediction score through the output layer of the joint ANCF model;

[0241] Among them, the joint prediction score The calculation expression is:

[0242] ;

[0243] In the formula, represents the joint prediction score, Represents the weight matrix of the output layer of the joint model.

[0244] T6: Based on the obtained joint prediction score, the joint ANCF model is trained and optimized to generate a product recommendation list and recommend items to users;

[0245] T61: Based on the obtained joint prediction score, the binary cross entropy loss function is iterated using gradient descent to obtain the optimal training result;

[0246] Among them, the calculation expression of the binary cross entropy loss function is:

[0247] ;

[0248] T62: Run the joint ANCF model for training optimization, and obtain the optimal HR and NDCG data while the loss function gradient decreases

[0249] Among them, the calculation expression of HR index is:

[0250] ;

[0251] In the formula, Indicates Whether the item clicked by the user is in the recommendation list, that is, 1 is used to indicate if it is in the list, and 0 is used to indicate if it is not in the list; Indicates the total number of users;

[0252] The calculation expression of NDCG index is:

[0253] ;

[0254] In the formula, ;

[0255] In the formula, ; ; Indicates the item score; Indicates item score Sort from largest to smallest and take the first A collection of results.

[0256] Example 3

[0257] This embodiment provides a neural collaborative filtering recommendation method combining an attention mechanism and an autoencoder, which differs from Embodiment 2 in that:

[0258] In this embodiment, five different data sets are collected for experiments: ml-1m and ml-100k from the Movielens movie website, ABaby and AMusic from the Amazon e-commerce website, and short film data Filmtrust.

[0259] By conducting experiments on the collected data of different data sets and comparing them with the existing ItemKNN, DMF, NeuMF, and GMF algorithms, the accuracy and effectiveness of the algorithms were evaluated using the HR and NDCG indicators, and the experimental results shown in Table 1 were obtained.

[0260] Table 1 Evaluation results of different algorithms on different datasets

[0261] ;

[0262] As shown in Table 1, the algorithm described in this application has better filtering effects in ml-1m, ml-100k, and Filmtrust datasets, as evaluated by HR and NDCG indicators. It is second only to the GMF algorithm on the ABaby dataset, and the HR indicator also has better filtering effects on the AMusict dataset. Therefore, the neural collaborative filtering algorithm of the marriage attention mechanism and autoencoder proposed in this application has better effects in recommendation.

[0263] like Figure 3 As shown, the ANCF algorithm proposed in this application converges downward during the iteration process, which means that the loss function has a better data fitting effect during the gradient descent process.

[0264] like Figure 4 and Figure 5 As shown in the figure, on the AMusict, Filmtrust, and ml-100k datasets, when different negative sampling num_negs are selected, the changes in the HR index and the NDCG index are relatively stable, which means that the ANCF algorithm proposed in this application is more robust under different negative sampling conditions and can adapt to actual data in different scenarios and with different interaction sample ratios.

[0265] like Figure 6 As shown in the figure, on the ml-100k dataset, when the negative sampling num_neg is fixed to 4, when the number of hidden factors in the hidden layer is different, the indicator change chart of HR indicator and NDCG shows that the best effect is achieved when the number of hidden layer factors is 128.

[0266] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0267] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A neural collaborative filtering recommendation method combining attention mechanism and autoencoder, characterized in that: The steps include: Collect user and product information data and construct a user-item interaction matrix; Based on the user-item interaction matrix, obtain the user feature vector and the item feature vector; Input the user feature vector and the item feature vector into the deep matrix decomposition model based on the attention mechanism and the autoencoder respectively to obtain the first prediction vector; The user feature vector and the item feature vector are respectively input into the multi-layer perceptron model based on the attention mechanism and the autoencoder to obtain the second prediction vector; Based on the preset model weights, the deep matrix decomposition model and the multi-layer perceptron model are combined, and the first prediction vector and the second prediction vector are fused to obtain a joint prediction score; Based on the obtained joint prediction score, a product recommendation list is generated to recommend items to users; The user feature vector and the item feature vector are input into the deep matrix decomposition model based on the attention mechanism and the autoencoder to obtain the first prediction vector, which specifically includes the following steps: The user feature vector is input into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the user's first decoding vector; and the obtained user's first decoding vector and the user feature vector are jointly input into the multi-layer perceptron for learning to obtain the user's potential feature vector; The item feature vector is input into the attention layer of the deep matrix decomposition model for encoding and decoding to obtain the first decoding vector of the item; and the obtained first decoding vector of the item and the user feature vector are jointly input into the multi-layer perceptron for learning to obtain the potential feature vector of the item; The dot product calculates the similarity between the user's potential feature vector and the item's potential feature vector, and combines the corresponding weight matrix and activation function to obtain the first prediction vector; The user feature vector and the item feature vector are respectively input into the multi-layer perceptron model based on the attention mechanism and the autoencoder to obtain the second prediction vector, which specifically includes the following steps: The parameter matrix of the linear embedding layer in the multi-layer perceptron model is used to transform the user feature vector into a user linear feature vector and the item feature vector into an item linear feature vector; The user linear feature vector and the item linear feature vector are respectively input into the attention layer of the multi-layer perceptron model for encoding and decoding, and the user second decoding vector and the item second decoding vector are respectively obtained; The user's second decoding vector and the item's second decoding vector are combined and used as the input vector of the multi-layer perceptron. The second prediction vector is obtained by combining the corresponding weight matrix and activation function.

2. According to claim 1, a neural collaborative filtering recommendation method combining attention mechanism and autoencoder is characterized in that: The process of collecting user and product information data and constructing a user-item interaction matrix specifically includes the following steps: Collect information data of user ID, item ID, and user's rating of items respectively, and pre-process the data; Based on the preprocessed data, the interaction between the user and the item is obtained, and "1" is used to indicate that there is interaction between the user and the item, and "0" is used to indicate that there is no interaction between the user and the item; The user-item interaction matrix Y is constructed by using rows to represent user IDs, columns to represent item IDs, and the values ​​in the matrix to represent the user's interaction with the items.

3. According to claim 2, a neural collaborative filtering recommendation method combining attention mechanism and autoencoder is characterized in that: Based on the user-item interaction matrix, the process of obtaining the user feature vector and the item feature vector specifically includes: According to the rows in the user-item interaction matrix, the user feature vectors corresponding to the user ID are obtained, that is, Represents the user feature vector of the Uth row in the user-item interaction matrix Y; According to the columns in the user-item interaction matrix, the item feature vectors corresponding to the item ID are obtained respectively, that is, represents the first i The user feature vector of the column.

4. According to claim 3, a neural collaborative filtering recommendation method combining attention mechanism and autoencoder is characterized in that: The calculation expression of the user's first decoding vector or the item's first decoding vector is: ； In the formula, Decode the vector for the user first; is the input vector of the input layer in the deep matrix decomposition model, that is, , is the transposed matrix of the feedforward attention layer weight matrix, is the user-item interaction matrix The user ID is The transposed vector of to the user feature vector; is the attention ratio of the attention layer, i.e. , is the Softmax activation function, is the transposed matrix of the weight matrix of the BP neural network, is the bias vector of the BP neural network; User latent feature vector Or the item latent feature vector The calculation expression is: ； In the formula, is the user's potential feature vector; is the item’s potential feature vector; is the activation of the Xth layer of the BP neural network; is the ReLU activation function; is the transposed matrix of the X-th layer weight matrix of the BP neural network; is the bias vector of the Xth layer of the BP neural network; Similarity between user latent feature vector and item latent feature vector The calculation expression is: ； The first prediction vector The calculation expression is: ； In the formula, Similarity scores for users and items in deep matrix factorization models; is the Sigmod activation function; is the weight matrix of the output layer in the deep matrix factorization model.

5. According to claim 4, a neural collaborative filtering recommendation method combining attention mechanism and autoencoder is characterized in that: User linear feature vector The calculation expression is: ； In the formula, is the transposed matrix of the parameter matrix of the relinear embedding layer of the multi-layer perceptron model; Item linear feature vector The calculation expression is: ； In the formula, The transposed matrix of the parameter matrix of the relinear embedding layer of the multi-layer perceptron model; User second decoding vector The calculation expression is: ； In the formula, A second decoding vector for the user; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ; Item second decoding vector The calculation expression is: ； In the formula, is the second decoding vector of the item; is the input vector of the input layer in the multi-layer perceptron model, that is ; is the attention ratio of the attention layer, i.e. ; The input vector of the multilayer perceptron for: ； In the formula, is the input vector of the multilayer perceptron; The second prediction vector The calculation expression is: ； In the formula, Similarity scores for users and items in the multi-layer perceptron model; is the Sigmod activation function; is the weight matrix of the input layer in the multi-layer perceptron model; is the output vector of the Yth layer of the deep network, that is , is the transposed matrix of the Y-th layer weight matrix of the BP neural network; is the bias vector of the Yth layer of the BP neural network.

6. According to claim 5, a neural collaborative filtering recommendation method combining attention mechanism and autoencoder is characterized in that: The process of combining the deep matrix decomposition model and the multi-layer perceptron model based on the preset model weights and fusing the first prediction vector and the second prediction vector to obtain a joint prediction score specifically includes the following steps: The deep matrix decomposition model and the multi-layer perceptron model are combined to build a joint model for training, and the model weights of both are set to 1 / 2; The first prediction vector output by the deep matrix decomposition model and the second prediction vector output by the multi-layer perceptron model are fused, and the corresponding weight matrix and activation function are combined to obtain a joint prediction score; Joint prediction scoring The calculation expression is: ； In the formula, represents the joint prediction score, Represents the weight matrix of the output layer of the joint model.

7. The neural collaborative filtering recommendation method combining attention mechanism and autoencoder according to claim 6, characterized in that: The process of training the joint model includes the following steps: Based on the interaction between users and items, and assuming that the interaction between users and items follows Bernoulli distribution, a likelihood function is defined to predict the interaction probability between users and items. Among them, the likelihood function Defined as: ； In the formula, For actual rating; Represents all the interactions between users and items that can be observed in the sampled data. Indicates the interactions between users and items that cannot be observed in the sampled data, that is, the number of negative samples; The binary cross entropy loss function is used to calculate the loss value of the joint model; Among them, the calculation expression of the binary cross entropy loss function is: ； The HR index and NDCG index are used to evaluate the performance of the joint model; Among them, the calculation expression of HR index is: ； In the formula, Indicates Whether the item clicked by the user is in the recommendation list, that is, 1 is used to indicate if it is in the list, and 0 is used to indicate if it is not in the list; Indicates the total number of users; The calculation expression of NDCG index is: ； In the formula, ; ; Indicates the item score; Indicates item score Sort from largest to smallest and take the first A collection of results.

Images