A Sequential Recommendation Method Based on Users' Long-Term and Short-Term Preferences

Abstract

The present invention discloses a sequential recommendation method based on users' long-term and short-term preferences. Traditional recommendation models use collaborative filtering algorithms to model users' latent interests. However, users' interests are often complex, changeable, and time-varying. A single model cannot accurately model users' interest characteristics. To solve this problem, according to users' historical sequential data, the present invention uses a gated recurrent unit integrated with an attention mechanism to model users' short-term interests. At the same time, a neural collaborative filtering latent factor model is adopted to extract users' long-term interests based on users' global information. Different weights are assigned to the long-term and short-term interests through the attention mechanism to obtain the final recommendation result.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer recommendation systems and relates to a sequence recommendation method based on a recurrent neural network. Background Art

[0002] Currently, in the field of recommendation systems, personalized recommendation systems have always received more attention. A personalized recommendation system generally deeply analyzes and mines factors such as user characteristics and interests by using user historical behavior data, and matches information or services that meet the needs for users based on this. Its most important feature lies in being able to fully adapt to the situation where user needs are unclear and complex, and being able to use the user's historical data to construct a reasonable algorithm to capture the user's interests, so as to find items or information that the user is interested in from a vast amount of information for the user to make decisions. As the interaction between consumers and relevant websites becomes more and more frequent, people's dependence on recommendation technology is also getting stronger. Therefore, how to mine the user's interests from the user's historical behavior data and provide accurate recommendation services for them has become an urgent problem to be solved at present. However, similar algorithms often ignore the problem of user interest drift. In daily life, user interests generally do not maintain a very stable state, especially in some frequently consumed goods, such as Weibo, music, and e-commerce websites. Among them, the time-series dynamic factor has always played an extremely important role in the actual application of recommendation systems, such as the product recommendation of Amazon, the film and television work recommendation of Netflix, and the news and video recommendation of Google and other recommendation systems. When they make recommendations, they are extremely sensitive to the changes in the user's dynamic interests, and the changes in the user's dynamic interests are hidden in the user's historical behavior sequence data. Therefore, the sequence recommendation system occupies an extremely important position in the development of the recommendation system at the present stage.

[0003] Most traditional recommendation algorithms are optimized from content-based recommendations and social network-based recommendations. Among them, the interaction behaviors of users with goods all exist in the form of independent information. In real life, however, the successive behaviors of users are all related, and even cause and effect each other. In a real scenario, the shopping behaviors of users usually occur in sequence rather than in isolation. The sequential dependence relationship of interactions usually exists in transaction data, and the interactions between users and items usually occur in the context of a certain order. Different contexts usually lead to users interacting with different items, but this dependence relationship and interaction cannot be well captured by conventional content-based recommendations or collaborative filtering recommendation algorithms. As time goes by, the preferences of users and the popularity of goods are both dynamically changing, and this dynamic change is very important for analyzing user preferences.

[0004] The model structure of a Recurrent Neural Network (RNN) consists of an input part, a hidden layer part, and an output part. It usually takes a sequential sequence as input and calculates the output result through the hidden layer structure. The output result of each layer contains the content of the previous layer. In the recommendation model based on RNN, the recommendation system often uses Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) to model the user's historical behavior sequence. Considering the good effect of the Gated Recurrent Unit in sequence recommendation, this invention uses the Gated Recurrent Unit to model the user's short-term interest and introduces an attention mechanism to control the update gate of the GRU network.

[0005] The attention mechanism is also one of the main methods for processing sequence information. Its core idea is to model the internal structure of the sequence using the complete context information while preserving the temporal characteristics of the information. When processing the complete commodity sequence, the context of the input information at each moment is known, and the model can utilize all the information of the sequence input during the decoding process. While calculating the output of the current layer, the RNN only uses the state of the previous moment. Summary of the Invention

[0006] The purpose of this invention is to solve the problem in the above prior art that it is unable to effectively capture the changes in user interests and model the user's dynamic interests. This invention adopts a Gated Recurrent Unit with an attention mechanism to obtain the user's short-term interest, uses a Multi-Layer Perceptron (MLP) to model the user's long-term interest based on the user's global information, and finally assigns different weights to the long-term and short-term interests through the attention mechanism to obtain the final recommendation result.

[0007] In view of this, the technical solution adopted by this invention is: a sequence recommendation method based on user's long-term and short-term preferences, which is characterized by inputting the user's historical behavior sequence into the model and obtaining the recommendation list finally recommended to the user through an embedding layer, an interest extraction layer, and an interest fusion layer, including the following steps:

[0008] S1. Obtain the user's historical behavior sequence and introduce item2vec to generate the user vector representation through item embedding.

[0009] S2. Adopt a Gated Recurrent Unit to model the user's short-term interest and introduce an attention mechanism to control the update gate of the GRU network.

[0010] S3. Use an MLP to model the user's long-term interest and find the user's high-order features in the latent factor vector space.

[0011] S4. Aggregate the long-term interest and the short-term interest based on the attention mechanism and calculate the corresponding recommendation result using the fused long-term and short-term interests.

[0012] Furthermore, in view of the fact that the one-hot encoded item embedding method cannot optimize a large amount of high-dimensional data well and will affect the model performance, item2vec is used for item embedding. Here, is used to represent the user interaction sequence of the embedding vector. is an element in the embedding vector Q u . represents that user u had an interaction behavior with item at time

[0013] Furthermore, the process of extracting the short-term interest of users by the short-term interest module is as follows. The attention mechanism is introduced to control the update gate of the GRU network. First, the embedding vectors of the historical behavior sequence and the target item vector need to be input into the attention network. The attention network calculates the correlation weight w between the two and passes it to the GRU network. The calculation formula of the attention network is as follows:

[0014]

[0015]

[0016] where a t is the output weight, indicating the degree of correlation between the current historical behavior and the target item. The larger this value is, the higher the correlation between the two; is the transformation vector that projects the hidden layer to the output weight; ⊙ is the inner product operation; β is the smoothing exponent used to adjust the softmax function, f(q, p) is the similarity, W a is the weight matrix, q t is the input of the GRU network, p is the target item, and b a is the bias vector.

[0017] The difference between the AGRU with the attention mechanism proposed by this method and the traditional GRU lies in that the attention mechanism is introduced into the update gate. The specific calculation is as follows

[0018] Specifically:

[0019]

[0020]

[0021] In the formula is the update gate with the attention mechanism introduced, u′ t is the original update gate, h′ t-1 is the hidden state in the AGRU network; is the short-term interest of the user.

[0022] Furthermore, the long-term interest module uses an MLP to model the user's historical behavior sequence. In this module, all users and items are mapped into the same latent factor space, and each user u and item i are respectively associated with a latent factor vector, that is and q i , Each element in represents the degree of the user's preference for the item, and q i contains the latent features of item i, and q i are the latent vectors of the user and the item respectively. The specific formula is as follows:

[0023]

[0024] In the formula, a * , w * , b * are the activation function, weight matrix, and bias function of the *-th layer of the MLP respectively, is the long-term interest of the user.

[0025] Furthermore, the present model designs a method to aggregate the long-term and short-term interests in an adaptive manner and calculate the corresponding recommendation results using the fused long-term and short-term interests. In this method, the user interest weight depends on the context, and based on the attention mechanism, the following formula is obtained

[0026]

[0027] In the formula, h T is the mapping from the hidden layer to the attention weight; ReLU(·) is the activation function; is the candidate of the attention network; is the user interest sequence obtained in the previous two stages; w and b are the weight and bias vector of the matrix respectively;

[0028]

[0029]

[0030] represents the correlation between the candidate of the attention network and the interest sequence. When the candidate has a high correlation with the interest sequence, the influence of this option on the recommendation should be increased. α jt is the output weight, is the similarity, H u is the target item sequence, is the target item.

[0031] The present invention uses the softmax function to convert the weight of the candidate into a probability expression, and shows how the system allocates short-term interests and long - term interests on the influence of recommendation results.

[0032]

[0033] In the formula represents the preference obtained in the user's short - term sequence stage, represents the long - term preference of the user, and the parameter obtained from the previous formula adjusts the weights of preferences at different stages in the final prediction. Finally, the final recommendation list p is obtained at this stage u .

[0034] The beneficial technical effects of the present invention are as follows:

[0035] (1) The present invention belongs to the field of sequential recommendation algorithms and can perform dynamic item - personalized recommendations for users in a recommendation system.

[0036] (2) A novel gated neural network for extracting user short - term interests by introducing an attention mechanism is proposed. Using this network structure can better dynamically capture user short - term preferences.

[0037] (3) A method for aggregating user long - term and short - term preferences by introducing an attention mechanism is proposed, which can effectively improve the accuracy of recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is the algorithm flow of the sequential recommendation model based on user long - term and short - term preferences of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and detailedly described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention.

[0040] Embodiment 1

[0041] As Figure 1 , the user historical behavior sequences are respectively the user short - term historical behavior sequence and the long - term historical behavior sequence. The embedding layer obtains the item embedding vector representation through item2vec; the short - term interest module models the user short - term interest through a gated recurrent unit introducing an attention mechanism; the user long - term interest is input into the MLP to find high - order features; the long - term and short - term interests obtained from the long - term interest module and the short - term interest module are respectively input into the adaptive fusion module introducing an attention mechanism for calculation to obtain the final recommendation result. Specifically, it includes the following steps:

[0042] S1. Obtain the user historical behavior sequence and introduce item2vec to generate the user vector representation by item embedding.

[0043] S2. Model the short-term interests of users using a gated recurrent unit, and introduce an attention mechanism to control the update gate of the GRU network.

[0044] S3. Use an MLP to model the long-term interests of users and find the high-order features of users in the latent factor vector space.

[0045] S4. Aggregate the long-term and short-term interests based on the attention mechanism, and calculate the corresponding recommendation results using the fused long-term and short-term interests.

[0046] In this example, the dataset used is the MovieLens-1M dataset collected and processed by the GroupLens Research Project of the Department of Computer Science and Engineering at the University of Minnesota. This dataset includes 836,478 rating data of 6,039 users for 3,628 movies, and each user has rated at least 20 movies. For experimental needs, this paper converts the explicit data of ratings into a binary classification standard, using the implicit data of 0 or 1 for users' ratings of movies to represent whether the user has an interaction history with the movie.

[0047] In this example, the dataset is randomly divided. 70% of the dataset is used for training and 30% for testing. In this example, movies with fewer than 5 interactions in the dataset are deleted.

[0048] Specifically, the short-term preference behavior sequence in the example of the present invention is the last 5 interaction behaviors of the user, and the long-term preference is all the historical behavior sequences of the user.

[0049] The present invention proposes a sequential recommendation method based on users' long-term and short-term preferences, and obtains good recommendation results through experiments on real datasets.

Claims

1. A sequential recommendation method based on users' long-term and short-term preferences, characterized in that Input the user's historical behavior sequence into the model, and obtain the final recommended list for the user through the embedding layer, interest extraction layer, and interest fusion layer, including the following steps: S1. Obtain the user's historical behavior sequence, and introduce item2vec to generate the user vector representation through item embedding; S2. Use a gated recurrent unit to model the user's short-term interest, and introduce an attention mechanism to control the update gate of the GRU network; To introduce the attention mechanism to control the update gate of the GRU network, it is necessary to first input the embedding vector of the historical behavior sequence and the target item vector into the attention network. The attention network calculates the correlation weight w between the two and passes it to the GRU network. The calculation formula of the attention network is as follows: where a t is the output weight, representing the degree of relevance between the current historical behavior and the target item. The larger this value, the higher the correlation between the two; is the transformation vector that projects the hidden layer onto the output weight; ⊙ is the inner product operation; β is the smoothing exponent used to adjust the softmax function; The specific calculation of the update gate is as follows: In the formula is the updated gate introducing the attention mechanism, u′ t is the original updated gate, h′ t-1 is the hidden state in the AGRU network; is the short-term interest of the user; S3. Use an MLP to model the user's long-term interest and find the user's high-order features in the latent factor vector space; The long-term interest module uses an MLP to model the user's historical behavior sequence. In this module, all users and items are mapped to the same latent factor space, and each user u and item i are respectively associated with a latent factor vector, that is and q i , Each element in represents the degree of user preference for the item, and q i contains the latent features of item i, and q i are the latent vectors of the user and the item respectively, and the specific formula is as follows: where a * , W * , b * are respectively the activation function, weight matrix, and bias function of the *-th layer of the MLP, is the user's long-term interest; S4. Aggregate the long-term interest and short-term interest based on the attention mechanism, and calculate the corresponding recommendation results using the fused long-term and short-term interests.

2. The sequence recommendation method based on user's long-term and short-term preferences according to claim 1, wherein: Use the item2vec for item embedding, where is used to represent the user interaction sequence of the embedding vector.

3. The sequence recommendation method based on user's long-term and short-term preferences according to claim 1, characterized in that: Aggregate the long-term and short-term interests in an adaptive manner, and calculate the corresponding recommendation results using the fused long-term and short-term interests. The user interest weight depends on the context, and the following formula is obtained based on the attention mechanism where h T is the mapping from the hidden layer to the attention weights; ReLU(·) is the activation function; is the candidate of the attention network; are the user interest sequences of the two previous stages obtained; w and b are the weight and bias vector of the matrix respectively; Indicates the correlation between the candidates of the attention network and the interest sequence, uses the softmax function to convert the weights of the candidates into a probability expression, and shows how the system allocates short-term interest and long-term interest on the recommendation results; where represents the preference obtained in the user's short-term sequence stage, represents the user's long-term preference, and the final recommendation list is obtained in this stage.

Images