A recommendation algorithm based on serial auto-encoder

Abstract

The application discloses a recommendation algorithm based on serial auto-encoders, comprising 1) merging item-based rating information and user and item interaction auxiliary information into an auto-encoder for reconstruction output, obtaining a feature representation of the reconstruction output through traditional auto-encoder learning, using auxiliary information to help the original rating matrix to reconstruct, and reducing the loss of effective information; 2) designing a serial connection method of auto-encoders, obtaining the reconstruction output generated by the first auto-encoder, inputting the reconstruction part of the output to the original rating matrix into the second auto-encoder, comparing the output of the second auto-encoder, i.e., a predicted rating matrix, with the original rating matrix, and calculating the prediction accuracy. The application can utilize attribute information of items, process the attribute information through auto-encoders, use the attribute information as extended features of recommendation, and achieve the purpose of more accurate recommendation for users.

Description

Technical Field

[0001] This invention relates to the field of personalized data recommendation research, and in particular to a recommendation algorithm based on a serial autoencoder. Background Technology

[0002] In recent decades, the rapid development of the internet has led to a massive amount of information, often causing user confusion and difficulty in finding target content. Recommendation systems, as information search and filtering tools, can effectively alleviate information overload, provide personalized recommendations, and help users make decisions. In recent years, with the increasing demand for high-quality, personalized recommendations, recommendation systems have received extensive research in e-commerce, knowledge engineering, and personalized recommendation.

[0003] Among various recommendation methods, collaborative filtering has achieved superior performance over the past few decades. It generates a user-item co-occurrence matrix based on users' historical behavior and makes recommendations by calculating the similarity between users and items. Despite the success of collaborative filtering, recent studies have revealed limitations such as handling data sparsity and poor generalization ability, restricting further development. To alleviate these problems, Koren et al. proposed a matrix factorization method that decomposes the co-occurrence matrix in collaborative filtering into user and item matrices and uses the inner product of the implicit vectors of users and items for ranking recommendations.

[0004] Recent research has shown that deep neural networks can learn more abstract and higher-level feature representations, which has led to significant progress in improving recommendation performance. Compared with traditional recommendation methods, deep learning-based recommendation methods have stronger abstract representation learning capabilities and can uncover more hidden features behind the data. Furthermore, deep learning models are highly flexible in structure, allowing for adjustments to the model structure based on specific scenarios. For example, He et al. proposed a general recommendation framework called neural network-based collaborative filtering. By transforming the dot product operation between user and item vectors in traditional matrix factorization methods into interoperability with neural networks, the expressive power of the model is enhanced.

[0005] Among all recommendation methods based on deep neural networks, many frameworks are implemented on top of autoencoder models, which are among the most successful deep neural networks and have recently been actively adopted as collaborative filtering models. For example, Suvash et al. proposed an autoencoder-based recommendation system called Autorec. Autorec encodes representations of users or items and leverages the generalization ability of autoencoders to make recommendations. For a simple single-hidden-layer neural network structure, it allows for fast training and deployment. Following this approach, Zhang et al. proposed a hybrid recommendation model based on semi-autoencoders. This method utilizes content information and learned nonlinear features to generate personalized recommendations, achieving good results. However, due to the unequal dimensionality of neurons in the input and output layers, effective information is lost, inevitably affecting recommendation performance. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a recommendation algorithm based on a serial autoencoder. This algorithm can utilize the interaction information between users and movies and knowledge graphs to expand the features of movie information and serially connect the semi-autoencoder and the autoencoder to make more accurate recommendations for users.

[0007] The objective of this invention is achieved as follows: a recommendation algorithm based on a serial autoencoder, comprising the following steps:

[0008] 1) Integrate item-based rating information and auxiliary information from user-item interactions into the autoencoder for output reconstruction. Learn the feature representation of the reconstructed output through traditional autoencoder learning, and use auxiliary information to help reconstruct the original rating matrix, reducing the loss of effective information;

[0009] 2) Design a serial connection method between autoencoders. Obtain the reconstruction output generated by the first autoencoder and input the reconstruction part of the original scoring matrix into the second autoencoder. Compare the output of the second autoencoder, i.e. the predicted scoring matrix, with the original scoring matrix to calculate the prediction accuracy.

[0010] As a further limitation of the present invention, step 1) specifically includes:

[0011] Step 1.1) By obtaining user ratings and item attribute information, the item attribute information is one-hot encoded, and the encoded vector is used as the extended feature for recommendation.

[0012] Step 1.2) Based on the user and item rating matrices and item attribute information obtained in Step 1.1), an AutoEncoder model is introduced to process the observed user and item rating vectors r iand the attribute vector a of the item i Combining these, the input to the first autoencoder is defined as con(r) i ;a i The calculation formula is shown in (1):

[0013] con(r i ;a i ) = connection of r i and a i (1)

[0014] In addition, R I This represents the item-based rating vector, A. I Represents the attribute vector of all items, con(R) I A I )∈R n×(m+k) This represents the connection between the rating vector and the item vector;

[0015] Step 1.3) Connect the rating vector and item vector obtained in Step 1.2) con(R) I A I ); will con(R) I A I The input data is fed into the first autoencoder model, and the decompressed reconstructed output is obtained through the compression and decompression operations of the first autoencoder; the encoding layer of the autoencoder is defined as shown in formula (2):

[0016] R′1=g(f(con(R I A I (2)·W1+b1)·W′1+b′1)

[0017] In the encoding and decoding process of the autoencoder, W1∈R (m+k)×h and W′1∈R h×(m+k) Represents the weight matrix; b1∈R n×h and b′1∈R n×(m+k) The bias vector is represented by f and g, which represent nonlinear activation functions.

[0018] As a further limitation of the present invention, step 2) specifically includes:

[0019] A second autoencoder model is used to learn the reconstructed part of the original scoring information, defined as sub(R′1); the encoding and decoding layers of the second autoencoder are represented as shown in formulas (3) and (4):

[0020] ξ2=f(W2·sub(R′1)+b2) (3)

[0021] R′2=g(W′2·ξ2+b′2)=g(W′2·f(W2·sub(R′1)+b2)+b′2) (4)

[0022] Wherein, the weight matrices and bias vectors of the encoding and decoding layers are W2∈R t×n b2∈R t×m and W′2∈R n ×t b′2∈R n×m Adding L2 norm regularization terms to the weight matrices W2 and W′2 in the objective function makes the model's decoding more closely approximate the real data. The L2 norm regularization term is represented as shown in formula (5):

[0023]

[0024] Furthermore, in the two autoencoder models based on serial autoencoders, the optimal value is found through the stochastic gradient descent algorithm SGD. Therefore, the final objective function is shown in Equation (6):

[0025] J item =||(R′2-R)|| 2 +αJ r (6)

[0026] Here, α is a trade-off parameter that controls the balance of regularization terms in order to minimize the difference between the original input item-based rating matrix R and the predicted output rating matrix R′2. When the model converges, the output layer of the serial autoencoder is the matrix R′2 used for prediction, and the result of R′2 is used as the basis for recommendation.

[0027] The present invention adopts the above technical solution, and compared with the prior art, the beneficial effects are as follows: 1) The present invention uses an autoencoder model to learn the feature representation of the interaction information between users and items. The autoencoder model has the characteristics of fast convergence speed, no need for labels and good effectiveness, which makes the method more practical.

[0028] 2) This invention uses knowledge graphs to extend the features of item information. At the same time, the additional features obtained by extension are represented by low-dimensional features extracted by an autoencoder. The original user item rating matrix and the original features are input into the autoencoder, so that they can be applied to the recommendation model more conveniently and flexibly.

[0029] 3) This invention designs a serial connection between traditional autoencoders to learn better feature representations, thereby improving the accuracy of model recommendations. Attached Figure Description

[0030] Figure 1 Overall framework diagram of the present invention.

[0031] Figure 2 A schematic diagram of the self-encoder model structure in this invention.

[0032] Figure 3 A schematic diagram of the semi-autoencoder model structure in this invention. Detailed Implementation

[0033] like Figure 1 The recommendation algorithm based on a serial autoencoder shown includes the following steps:

[0034] 1) The item-based rating information and the auxiliary information of user and item interaction are merged into the autoencoder for output reconstruction. The feature representation of the reconstructed output is obtained through traditional autoencoder learning. The auxiliary information is used to help reconstruct the original rating matrix and reduce the loss of effective information. The specific steps are as follows;

[0035] Step 1.1) First, by obtaining user ratings and item attribute information, the item attribute information is one-hot encoded, and the encoded vector is used as the extended feature for recommendation. For example, in the field of movie recommendation, the classic recommendation algorithm test datasets MovieLens 100K and MovieLens 1M are used to obtain the attribute information of the corresponding movies in the datasets, such as movie ratings, movie release dates, movie categories, etc.

[0036] Step 1.2) Next, based on the user and item rating matrices and item attribute information obtained in Step 1.1), an Autoencoder model is introduced to process the observed user and item rating vectors r i and the attribute vector a of the item i Combining these, the input to the first autoencoder can be defined as con(r) i ;a i The calculation formula is shown in (1):

[0037] con(r i ;a i ) = connection of r i and a i (1)

[0038] In addition, R I This represents the item-based rating vector, A. I This represents the attribute vector for all items. con(R) I A I )∈R n×(m+k) This represents the connection between the rating vector and the item vector;

[0039] Step 1.3) Connect the rating vector and item vector obtained in Step 1.2) con(R) I A I ); will con(R) I A I The input data is fed into the first autoencoder model, and the decompressed reconstructed output is obtained through the compression and decompression operations of the first autoencoder; the encoding layer of the autoencoder can be defined as shown in formula (2):

[0040] R′1=g(f(con(R I A I (2)·W1+b1)·W′1+b′1)

[0041] In the encoding and decoding process of the autoencoder, W1∈R (m+k)×h and W′1∈R h×(m+k) Represents the weight matrix; b1∈R n×h and b′1∈R n×(m+k) represents the bias vector; f and g represent non-linear activation functions, and this algorithm uses the identity and sigmoid activation functions respectively.

[0042] 2) Design a serial connection method between autoencoders. Obtain the reconstructed output of the first autoencoder and input the reconstructed portion of the original rating matrix into the second autoencoder. Compare the output of the second autoencoder (predicted rating matrix) with the original rating matrix to calculate the prediction accuracy, achieving more precise recommendations. The specific steps are as follows:

[0043] Step 1.3) observes that a portion of the output of the first autoencoder model is a reconstruction of the original rating information; therefore, in order to extract higher-dimensional feature representations from the reconstructed output, a second autoencoder model is used to learn the reconstructed portion of the original rating information, defined as sub(R′1); the encoding and decoding layers of the second autoencoder can be represented as shown in formulas (3) and (4):

[0044] ξ2=f(W2·sub(R′1)+b2) (3)

[0045] R′2=g(W′2·ξ2+b′2)=g(W′2·f(W2·sub(R′1)+b2)+b′2) (4)

[0046] Wherein, the weight matrices and bias vectors of the encoding and decoding layers are W2∈R t×n b2∈R t×m and W′2∈R n ×t ,b′2∈R n×m; To avoid overfitting of the model, the L2-norm regularization terms of weight matrices W2 and W′2 are added to the objective function, making the decoding of the model closer to the real data. The representation of the L2-norm regularization term is shown in formula (5):

[0047]

[0048] In addition, in the two autoencoder models based on the serial autoencoder, the stochastic gradient descent algorithm (SGD) is used to quickly find the optimal value. Therefore, the final objective function of the recommendation algorithm based on the serial autoencoder is shown in formula (6):

[0049] J item = ||(R′2 - R)|| 2 + αJ r (6)

[0050] where α is a trade-off parameter that controls the balance of the regularization term, and it is used to minimize the difference between the input original item-based rating matrix R and the predicted output rating matrix R′2. When the model converges, the output layer of the serial autoencoder is the matrix R′2 for prediction, and the result of R′2 is used as the basis for recommendation.

[0051] The present invention can be further illustrated by the following experiments:

[0052] To test the effectiveness of the present invention, movie recommendation is taken as an example. The classic movie datasets MovieLens100K and MovieLens 1M are selected to implement prediction on the two datasets respectively. Among them, the MovieLens 100K dataset includes 100,000 ratings of 1,682 movies by 943 users; the MovieLens 1M dataset includes 1,000,209 ratings of 3,706 movies by 6,040 users. The evaluation metrics of the experiments of the present invention are the mean absolute error (MAE) and the root mean square error (RMSE), and the calculation formulas are as follows. The smaller the MAE and RMSE values are, the better the performance of the recommendation system is.

[0053]

[0054]

[0055] where r u,i and respectively represent the original and reconstructed ratings of user u for movie i, and |TestSet| represents the entire test set.

[0056] To demonstrate the performance of the test results, classic recommendation algorithm test datasets, MovieLens 100K and MovieLens 1M, were used. Nonnegative matrix factorization (NMF), improved singular value decomposition (SVD++), meta-learning methods applied to recommendation systems (MetaHIN), graph-based recommendation systems (GraphRec), semi-autoencoder-based recommendation systems (HCRSA), and a knowledge graph combined with a two-layer autoencoder recommendation system (PRKG) were selected for comparison. Tables 1 and 2 show the prediction results of the mean absolute error (MAE) and root mean square error (RMSE) of our proposed algorithm on the two datasets. The experimental results show that the overall performance of our proposed recommendation algorithm based on a serial autoencoder is superior to other methods.

[0057] Table 1. Experimental results using the MovieLens 100K dataset.

[0058]

[0059] Table 2 Experimental results using the MovieLens 1M dataset

[0060]

[0061]

[0062] This invention utilizes the attribute features related to electrical items and integrates the original rating matrix information and item attribute information into a traditional autoencoder to obtain efficient feature representations. During compression, auxiliary information effectively assists in the reconstruction of the original rating matrix, thereby solving the data sparsity problem while effectively reducing the loss of key auxiliary information. Furthermore, by designing a serial connection between autoencoders, it learns more abstract and higher-level feature representations for personalized recommendations, enabling more accurate recommendations for users.

[0063] This invention is not limited to the above embodiments. Based on the technical solutions disclosed in this invention, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

Claims

1. A recommendation method based on a serial autoencoder, characterized in that, Includes the following steps: 1) Integrate item-based rating information and auxiliary information from user-item interactions into the autoencoder for output reconstruction. Learn the feature representation of the reconstructed output through traditional autoencoder learning, and use auxiliary information to help reconstruct the original rating matrix, reducing the loss of effective information; Step 1.1) By obtaining user ratings and item attribute information, the item attribute information is one-hot encoded, and the encoded vector is used as the extended feature for recommendation. Step 1.2) Based on the user and item rating matrices and item attribute information obtained in Step 1.1), an AutoEncoder model is introduced to process the observed user and item rating vectors r i and the attribute vector a of the item i Combining these, the input to the first autoencoder is defined as con(r) i ;a i The calculation formula is shown in (1): con(r i ;a i )=connection of r i and a i (1) In addition, R I This represents the item-based rating vector, A. I Represents the attribute vector of all items, con(R) I A I )∈R n ×(m+k) This represents the connection between the rating vector and the item vector; Step 1.3) Connect the rating vector and item vector obtained in Step 1.2) con(R) I A I ); will con(R) I A I The input data is fed into the first autoencoder model, and the decompressed reconstructed output is obtained through the compression and decompression operations of the first autoencoder; the encoding layer of the autoencoder is defined as shown in formula (2): R′1=g(f(con(R I ;A I )·W1+b1)·W′1+b′1) (2) In the encoding and decoding process of the autoencoder, W1∈R (m+k)×h and W′1∈R h×(m+k) Represents the weight matrix; b1∈R n×h and b′1∈R n×(m+k) The bias vector is represented by f and g, which represent nonlinear activation functions. 2) Design a serial connection method between autoencoders. Obtain the reconstruction output generated by the first autoencoder and input the reconstruction part of the original scoring matrix into the second autoencoder. Compare the output of the second autoencoder, i.e. the predicted scoring matrix, with the original scoring matrix to calculate the prediction accuracy. A second autoencoder model is used to learn the reconstructed part of the original scoring information, defined as sub(R′1); the encoding and decoding layers of the second autoencoder are represented as shown in formulas (3) and (4): ξ2=f(W2·sub(R′1)+b2) (3) R′2=g(W′2·ξ2+b2 ′ )=g(W′2·f(W2·sub(R ′ 1)+b2)+b′2) (4) Wherein, the weight matrices and bias vectors of the encoding and decoding layers are W2∈R t×n b2∈R t×m and W′2∈R n×t b′2∈R n×m Adding L2 norm regularization terms to the weight matrices W2 and W′2 in the objective function makes the model's decoding more closely approximate the real data. The L2 norm regularization term is represented as shown in formula (5): Furthermore, in the two autoencoder models based on serial autoencoders, the optimal value is found through the stochastic gradient descent algorithm SGD. Therefore, the final objective function is shown in Equation (6): J item =||(R′2-R)|| 2 +αJ r (6) Here, α is a trade-off parameter that controls the balance of regularization terms in order to minimize the difference between the original input item-based rating matrix R and the predicted output rating matrix R′2. When the model converges, the output layer of the serial autoencoder is the matrix R′2 used for prediction, and the result of R′2 is used as the basis for recommendation.

Images