A next item recommendation method integrating users' long-term and short-term preferences

Abstract

The present invention discloses a next item recommendation method integrating long-term and short-term preferences of users, comprising the following steps: generating initial embeddings of items for each item in a conversation data set including several historical conversations of each user and one current conversation, and dividing the conversation data set into a training set, a test set and a validation set; using the training set to construct an initial model of a next item recommendation method ILSTP integrating long-term and short-term preferences of users; using the training set and the validation set to adjust parameters of the initial model of ILSTP to obtain an optimized model of ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model; using the obtained optimized model of ILSTP to provide the next item recommendation for users in the conversation data set or in actual conversation data. The next item recommendation method provided by the present invention has the characteristics of high recommendation accuracy and has broad application prospects in many fields such as movies, music, online news, and e-commerce.

Description

Technical Field

[0001] The present invention belongs to the technical field of session-based recommendation systems, and relates to a next item recommendation method based on a deep neural network, and in particular to a next item recommendation method that integrates a user's long-term preference and short-term preference. Background Art

[0002] In the era of information overload and digital economy, recommender systems play an increasingly important role in informed consumption, services, and decision-making. With the continuous development of recommender system technology, session-based recommender systems, also known as conversational recommender systems, have become a new paradigm for recommender systems (see: Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Organ, Defu Lian: A Survey on Session-based Recommender Systems. ACM Comput. Surv. 54(7): 154: 1-154: 38 (2022). https: / / doi.org / 10.1145 / 3465401). Conversational recommender systems capture users' short-term, dynamic preferences based on their session information and provide timely and accurate recommendations. A session in a conversational recommender system is a non-empty bounded list of user-item interactions issued by a user in a continuous period of time. In a conversational recommendation system, given a user and a time range, all the user's sessions within this time range can be further divided into several historical sessions and a current session. The key to conversational recommendation systems is to learn dependencies within and / or between sessions to capture user preferences, thereby providing accurate recommendations to users. User preferences are divided into user's long-term preference and user's short-term preference. Long-term preference refers to the user's long-term and relatively fixed preference, which is often captured from the user's historical session information. Short-term preference describes the user's interest changes in a short period of time, which is often captured from the user's current session information. The interaction data involved in a recommendation system in a certain period of time usually includes users, items, user-item interactions and interaction timestamps; a certain amount of interaction data constitutes an interaction dataset.In a conversational recommendation system, an interaction dataset needs to be converted into a session dataset (sessiondataset) containing the user's historical sessions and current sessions. The conversion method and steps are as follows: first, the interaction data in an interaction dataset is grouped by user, and then the interaction data of each user is processed as follows: the interaction data is sorted in the order of the interaction timestamps in time, and the sorted interaction data is divided into several sessions at a certain time interval, and several items with relatively close interaction timestamps in time form a session; for the user's session data, the session composed of several items with the most recent and relatively close interaction timestamps in time is regarded as a current session of the user, and several groups of items with earlier and relatively close interaction timestamps in time are regarded as several historical sessions of the user; the session data of all users constitute a session dataset.

[0003] Next-item recommendation (or next interaction recommendation) is the main task of session recommendation. Next-item recommendation aims to recommend the next possible interaction (e.g., clicking a song, purchasing a product) of the user in the current session based on a given session context (i.e., a list of interactions that have occurred).

[0004] A conversational recommendation method (method or approach) is also called a conversational recommendation model (model). In recent years, conversational recommendation models based on deep neural networks have become the mainstream models in conversational recommendation. Depending on whether the conversational recommendation model captures the user's long-term preferences, conversational recommendation models based on deep neural networks are divided into two categories: deep neural network-based models capturing only users'short-term preferences and deep neural network-based models capturing both users'long-term and short-term preferences. Deep neural network-based models that capture users' long-term and short-term preferences utilize users' long-term preferences to improve the recommendation performance of the model. However, most of them have some shortcomings in preference mining and integration. For example, the HRNN model (see: Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi: Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks. RecSys 2017: 130-137. https: / / doi.org / 10.1145 / 3109859.3109896) uses a session-level RNN to capture users' short-term preferences, and then uses a user-level RNN to capture users' long-term preferences, ignoring the time interval information between user-item interactions.For example, the A-PGNN model (see: Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, and Liang Wang: Personalized Graph Neural Networks with Attention Mechanism for Session-Aware Recommendation. IEEE Trans. Knowl. Data Eng. 15 October 2020 (Early Access). https: / / doi.org / 10.1109 / TKDE.2020.3031329) only captures users' long-term and short-term preferences for recommendation by constructing user behavior graphs using the user's current session and historical sessions, and also ignores the time interval information between user-item interactions.

[0005] In order to overcome the above-mentioned deficiencies of the prior art in the relevant technical field, the present invention intends to use deep learning technology based on neural networks to construct a next item recommendation model that integrates users' long-term preferences and short-term preferences as a next item recommendation method. Such a recommendation method (model) can utilize the time interval information between user-item interactions to mine more accurate users' long-term preferences, and effectively integrate them with users' short-term preferences, thereby improving the accuracy of the next item recommendation.

[0006] Deep neural networks are widely used in speech recognition, computer vision, and recommendation systems due to their powerful learning and expression capabilities, and have achieved great success. The attention mechanism in the field of deep neural networks (see: Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio: Neural Machine Translation by Jointly Learning to Align and Translate. 3rd International Conference on Learning Representations, ICLR 2015, May 7-9, 2015, Conference Track Proceedings. http: / / arxiv.org / abs / 1409.0473) is similar to the selective visual attention mechanism of humans, which can ignore unimportant information from a large amount of important information, filter out a small amount of important information and focus on these important information. Self-attention (see: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N.Gomez, LukaszKaiser, Illia Polosukhin: Attention is All you Need. NIPS 2017: 5998-6008. http: / / papers.nips.cc / paper / 7181-attention-is-all-you-need) is an attention mechanism, which is mainly used to calculate the relative importance between items in a conversation sequence. The core idea of ​​the present invention is to capture the user's accurate long-term preferences by organically combining self-attention with a time interval embedding function and integrate it with the user's short-term preferences to achieve the next item (interaction) recommendation and improve the recommendation accuracy.

[0007] The following is a brief description of several prior arts adopted in the technical solution of the present invention. These prior arts are only several single technical features in the technical solution of the present invention. They constitute the technical solution of the present invention only after being organically combined with other technical features of the present invention in a non-obvious manner.

[0008] The item2vec embedding model (see: Oren Barkan, Noam Koenigstein: ITEM2VEC: Neuralitem embedding for collaborative filtering. MLSP 2016: 1-6. https: / / doi.org / 10.1109 / MLSP.2016.7738886 or Oren Barkan, Noam Koenigstein: Item2vec: NeuralItem Embedding for Collaborative Filtering. RecSys Posters 2016. http: / / ceur-ws.org / Vol-1688 / paper-13.pdf) uses the idea of ​​maximum likelihood estimation and uses logarithmic loss to embed items. This embedding model can embed items in the recommendation system into a low-dimensional space and obtain a vector representation of the item, which is convenient for the recommendation method (model) to perform calculations.

[0009] Gated graph neural networks (see: Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard S. Zemel: Gated Graph Sequence Neural Networks. ICLR (Poster) 2016. https: / / arxiv.org / abs / 1511.05493) is a representative graph neural network, which uses gated recurrent units as recurrent functions. During the learning process, it regards the neighbor node information of the node in the graph as input, the state of the node itself as the hidden state, and selectively memorizes the hidden information of the neighbor nodes and the hidden information of the node itself.

[0010] The Adam optimizer (see: Diederik P. Kingma, Jimmy Ba: Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015, May 7-9, 2015, Conference Track Proceedings. https: / / arxiv.org / abs / 1412.6980) is used to optimize the parameters of the conversational recommendation model.

[0011] Average pooling is used to average the features of multiple items in a session to obtain representative features of the session.

[0012] Max pooling is used to find the maximum value in each component of the representative features of multiple sessions and combine these maximum values ​​into a new feature vector.

[0013] Softmax is an activation function that is used to map the next item probability (initial value) of each item in the conversation dataset to an open interval (0, 1).

[0014] Soft attention is an attention mechanism that is often used in the field of recommender systems to calculate the similarity between items.

[0015] The present invention uses two performance evaluation indicators commonly used in the technical field to evaluate the accuracy of the recommendation model: recall rate (Recall@N) and mean reciprocal ranking (MRR@N) (see: Charu C. Aggarwal. Evaluating Recommender Systems. In: Rcommender Systems: The Textbook. Springer 2016, pp. 225-254. https: / / doi.org / 10.1007 / 978-3-319-29659-3_7), which are defined as follows:

[0016] (1) Recall rate: It is defined as the ratio of items in the recommendation list that are truly relevant to the user to all items that are truly relevant to the user. This indicator is defined by the following formula:

[0017]

[0018] Where N is the number of recommended items, m is the number of users, and C u (N) represents the top-N recommendation list of user u, GT u Represents the collection of all items that are actually relevant to the user.

[0019] (2) Average reciprocal ranking: It is defined as the sum of the reciprocal rankings of items that are truly relevant to the user in the recommendation list. This indicator is defined by the following formula:

[0020]

[0021] Among them, N represents the number of recommended items, m represents the number of users, and rank v represents the ranking of item v in the recommendation list of user u, GTu Represents the collection of all items that are actually relevant to the user.

[0022] In summary, in order to overcome the above-mentioned deficiencies in the prior art, namely: the conversational recommendation method / model based on deep neural network does not fully utilize the time interval information between user-item interactions and does not effectively integrate the long-term and short-term preferences of users, resulting in low recommendation accuracy, the present invention creatively combines self-attention with the time interval embedding function to mine the long-term preferences of users and effectively integrates them with the short-term preferences of users, thereby improving the accuracy of the next item recommendation. In short, the present invention provides a method for next item recommendation integrating long-term and short-term preferences of users (Integration of Users'Long-Term Preferences and Short-Term Preferences for Next Item recommendation, referred to as ILSTP). Summary of the invention

[0023] The technical problem to be solved by the present invention is to provide a next item recommendation method with high recommendation accuracy.

[0024] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0025] The present invention provides a next item recommendation method integrating a user's long-term preference and short-term preference, comprising the following steps:

[0026] Step S1: Generate initial embeddings for each item in a session dataset containing several historical sessions and one current session of each user, and divide the session dataset into a training set, a test set, and a validation set;

[0027] Step S2: constructing an initial model of the next recommendation method ILSTP that integrates the user's long-term preference and short-term preference using the training set;

[0028] Step S3: adjusting the parameters of the initial model of the ILSTP using the training set and the validation set to obtain an optimized model of the ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model;

[0029] Step S4: Using the obtained ILSTP optimization model, provide the next recommendation for the user in the session data set or the actual session data.

[0030] In the method, the initial embedding of generating items in step S1 further comprises:

[0031] Step S11: Use the item2vec embedding model to generate an initial embedding of each item in the conversation dataset, that is, represent an item as a d-dimensional vector, where the hyperparameter d∈{10, 20, ..., 120}.

[0032] In the method, step S2 further comprises:

[0033] Step S21: constructing a user preference capture layer in the initial model of the session recommendation method ILSTP, and using the layer to generate a user's long-term preference and a user's short-term preference;

[0034] Step S22: constructing a user preference integration layer in the initial model of the session recommendation method ILSTP, and using the layer to generate an integrated user preference;

[0035] Step S23: constructing a next item probability prediction layer in the initial model of the conversation recommendation method ILSTP, and using the layer to calculate the probability that each item in the conversation data set will become the user's next interactive item.

[0036] In the method, the step S21 further comprises:

[0037] Step S211: for a user in the conversation data set, all items in several historical conversations of the user are connected into a training sequence, and the interaction timestamps of the items are connected into a time series accordingly, and then the training sequence and the time series are truncated or expanded into a fixed-length training sequence with a length of l and a fixed-length time series with a length of l, wherein the hyperparameter l∈{50, 60, ..., 200}, wherein the truncation occurs when the actual length of the sequence exceeds l, and the expansion occurs when the actual length of the sequence is less than l;

[0038] Step S212: for the fixed-length training sequence, calculating the time interval Δt between any two items in the sequence, where the time interval is the absolute value of the difference between two interaction timestamps corresponding to the two items in the fixed-length time sequence;

[0039] Step S213: for the fixed-length training sequence, calculate the time interval embedding r between any two items in the sequence, the time interval embedding is calculated using a time interval embedding function, and the calculation method of the time interval embedding function is defined by the following formula:

[0040]

[0041] Where Δt is the time interval between any two items in a fixed-length training sequence, and σ is a sigmoid activation function;

[0042] Step S214: The layer number k of the K-layer self-attention and feedforward neural network is used as the loop variable of the counting control loop, whose initial value is 1, whose termination value is K, and whose incremental step is 1. The following steps (1) to (2) are repeated K times, where the hyperparameter K∈{1, 2, 3, 4}:

[0043] (1) For the fixed-length training sequence, calculate the k-th layer attention score between any two items v1 and v2 in the sequence And use a softmax function to Normalized to the k-th layer weight coefficient, as follows:

[0044] Based on the k-th layer embedding of items v1 and v2 and and the time interval between these two items is embedded in r 12 To calculate the attention score between these two items Among them, when k = 1, the first layer embedding of these two items and is the initial embedding of the item obtained in step S11, and the calculation method is defined by the following formula:

[0045]

[0046] Where T is the matrix transpose operation, W Q , is the weight matrix of the user preference capture layer, whose values ​​are determined by parameter learning, and the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item;

[0047] Get the above k-th layer attention score Then, a softmax function is used to convert Normalized to the k-th layer weight coefficient;

[0048] (2) For the fixed-length training sequence, calculate the k-th transformed embedding of l items in the sequence, as follows:

[0049] First, the k-th layer embeddings of all items in the sequence are weighted aggregated according to the k-th layer weight coefficient between any item in the sequence and all items in the sequence to generate the k-th layer aggregated embedding z of any item k ;

[0050] Then, a feed-forward neural network is used to k Transform to obtain the k-th transformed embedding f of the item k, the conversion method of the feedforward neural network is defined by the following formula:

[0051] f k =ReLU(z k W1+b1)W2+b2,

[0052] Where, W1, is the weight matrix of the user preference capture layer, b1, is the bias vector of the user preference capture layer, whose values ​​are determined by parameter learning, the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item, and ReLU is a nonlinear activation function;

[0053] Finally, after K repetitions of the above, a high-order embedding of the time interval perception of l items in the fixed-length training sequence is obtained;

[0054] Step S215: for the fixed-length training sequence, the time interval-aware high-order embeddings of l items in the sequence are grouped according to historical sessions, and a hybrid aggregator is used to generate a user's long-term preference, specifically including:

[0055] First, performing maximum pooling on the high-order embeddings of all items in several historical sessions of the user to generate embeddings of several historical sessions;

[0056] Then, average pooling is performed on the embeddings of the several historical sessions to generate a user's long-term preference;

[0057] Step S216: For a current session of the user, convert the current session into a session graph, which is defined as follows:

[0058] The session graph is a weighted directed graph, where each node in the graph represents an item in the current session, and each directed edge in the graph represents that the user interacts with the start item of the edge and then interacts with the end item of the edge in the current session. The weight of the edge is calculated by dividing the number of times the edge appears in the current session by the out-degree of the start item of the edge.

[0059] Step S217: for the session graph, a gated graph neural network is used to generate the embedding of each item in the session graph, and a local embedding and a global embedding of the current session are calculated, and then the two are concatenated and linearly transformed to generate the user's short-term preference, wherein the local embedding is the embedding of the last item in the current session, and the global embedding is calculated using soft attention, and the calculation process is as follows:

[0060] First, soft attention is used to calculate the weight coefficient between the embedding of the last item in the current session and the embedding of all items in the current session;

[0061] Then, the embeddings of all items in the current session are weighted and aggregated according to the calculated weight coefficients to generate a global embedding.

[0062] In the method, step S22 further comprises:

[0063] Step S221: using an attention aggregator to generate an integrated user preference, specifically including:

[0064] First, a long-term preference coefficient is calculated for the user's long-term preference using an attention mechanism;

[0065] Then, a short-term preference coefficient is calculated for the user's short-term preference using an attention mechanism;

[0066] Finally, the long-term preference and short-term preference of the user are weighted and aggregated according to the calculated long-term preference coefficient and short-term preference coefficient to generate an integrated user preference.

[0067] In the method, step S23 further comprises:

[0068] Step S231: performing vector dot product on the integrated user preferences and the initial embedding of each item in the conversation dataset to obtain prediction scores of these items;

[0069] Step S232: Use a softmax function to normalize the prediction score to generate the probability that each item in the conversation data set will become the user's next interaction item.

[0070] In the method, step S3 further comprises:

[0071] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the adjusted model of the ILSTP on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0072] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the length l of the fixed-length training sequence, the number of layers K of the self-attention and feedforward neural networks, the learning rate η, and the regularization coefficient λ. The search range of these hyperparameters is as follows: the range of d is the set {10, 20, ..., 120}, the range of η is the set {0.005, 0.001, 0.01, 0.05, 0.1}, and the range of λ is the set {10 -5 , 10 -4 , 10-3 , 10 -2 , 10 -1}, the range of K is the set {1, 2, 3, 4}, and the range of l is the set {50, 60, ..., 200};

[0073] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the ILSTP parameter-adjusted model;

[0074] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0075] Step S32: Determine whether the recommendation accuracy of the ILSTP model after parameter adjustment reaches a predetermined threshold, where the recommendation accuracy is the recall rate, and the threshold value range is a closed interval [0.15, 0.25]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence, otherwise terminate the parameter adjustment process to obtain the optimized model of ILSTP;

[0076] Step S33: Use the test set and two performance evaluation indicators, namely, recall rate and average reciprocal rank, to evaluate the recommendation accuracy of the obtained ILSTP optimization model.

[0077] The beneficial technical effects of the present invention mainly include three aspects: (1) The present invention provides a next item recommendation method with high recommendation accuracy; (2) The next item recommendation method ILSTP of the present invention has higher recommendation accuracy than the typical deep neural network-based models HRNN and A-PGNN that capture users' long-term preferences and short-term preferences. The reason is that the technical solution of the present invention fully utilizes the time interval information between user-item interactions in the user's session data by organically combining time interval embedding functions, self-attention and hybrid aggregators to mine more accurate user preferences; (3) The present invention has broad application prospects in many fields such as music, movies, online news, and e-commerce.

[0078] The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings. Additional aspects and advantages of the present invention will be partially given in the following description, which will become apparent from the following description or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic flow chart of a next item recommendation method integrating a user's long-term preference and short-term preference according to the technical solution of the present invention is shown;

[0080] Figure 2A model framework diagram of a next item recommendation method integrating a user's long-term preference and short-term preference according to the technical solution of the present invention is shown;

[0081] Figure 3 A flowchart of constructing an initial model in a next item recommendation method integrating a user's long-term preference and short-term preference according to the technical solution of the present invention is shown. DETAILED DESCRIPTION

[0082] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar concepts, objects, elements, etc. or concepts, objects, elements, etc. with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0083] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the present invention belongs and related fields. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0084] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0085] The present invention provides a next item recommendation method integrating the user's long-term preference and short-term preference, such as Figure 1 As shown, the following steps S1 to S4 are included:

[0086] Step S1: generating initial embeddings for each item in a session data set including several historical sessions of each user and one current session, and dividing the session data set into a training set, a test set, and a validation set; generating initial embeddings for the items in step S1 further includes:

[0087] Step S11: Use the item2vec embedding model to generate an initial embedding of each item in the conversation dataset, that is, represent an item as a d-dimensional vector, where the hyperparameter d∈{10, 20, ..., 120}.

[0088] Step S2: constructing an initial model of the next recommendation method ILSTP that integrates the user's long-term preference and short-term preference using the training set;

[0089] In this method, if Figure 2 and Figure 3As shown, the step S2 further comprises:

[0090] Step S21: constructing a user preference capture layer in the initial model of the session recommendation method ILSTP, and using the layer to generate a user's long-term preference and a user's short-term preference; the step S21 further includes:

[0091] Step S211: for a user in the conversation data set, all items in several historical conversations of the user are connected into a training sequence, and the interaction timestamps of the items are connected into a time series accordingly, and then the training sequence and the time series are truncated or expanded into a fixed-length training sequence with a length of l and a fixed-length time series with a length of l, wherein the hyperparameter l∈{50, 60, ..., 200}, wherein the truncation occurs when the actual length of the sequence exceeds l, and the expansion occurs when the actual length of the sequence is less than l;

[0092] Step S212: for the fixed-length training sequence, calculating the time interval Δt between any two items in the sequence, where the time interval is the absolute value of the difference between two interaction timestamps corresponding to the two items in the fixed-length time sequence;

[0093] Step S213: for the fixed-length training sequence, calculate the time interval embedding r between any two items in the sequence, the time interval embedding is calculated using a time interval embedding function, and the calculation method of the time interval embedding function is defined by the following formula:

[0094]

[0095] Where Δt is the time interval between any two items in a fixed-length training sequence, and σ is a sigmoid activation function;

[0096] Step S214: The layer number k of the K-layer self-attention and feedforward neural network is used as the loop variable of the counting control loop, whose initial value is 1, whose termination value is K, and whose incremental step is 1. The following steps (1) to (2) are repeated K times, where the hyperparameter K∈{1, 2, 3, 4}:

[0097] (1) For the fixed-length training sequence, calculate the k-th layer attention score between any two items v1 and v2 in the sequence And use a softmax function to Normalized to the k-th layer weight coefficient, as follows:

[0098] Based on the k-th layer embedding of items v1 and v2 and and the time interval between these two items is embedded in r 12 To calculate the attention score between these two items Among them, when k = 1, the first layer embedding of these two items and is the initial embedding of the item obtained in step S11, and the calculation method is defined by the following formula:

[0099]

[0100] Where T is the matrix transpose operation, W Q , is the weight matrix of the user preference capture layer, whose values ​​are determined by parameter learning, and the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item;

[0101] Get the above k-th layer attention score Then, a softmax function is used to convert Normalized to the k-th layer weight coefficient;

[0102] (2) For the fixed-length training sequence, calculate the k-th transformed embedding of l items in the sequence, as follows:

[0103] First, the k-th layer embeddings of all items in the sequence are weighted aggregated according to the k-th layer weight coefficient between any item in the sequence and all items in the sequence to generate the k-th layer aggregated embedding z of any item k ;

[0104] Then, a feed-forward neural network is used to k Transform to obtain the k-th transformed embedding f of the item k , the conversion method of the feedforward neural network is defined by the following formula:

[0105] f k =ReLU(z k W1+b1)W2+b2,

[0106] Where, W1, is the weight matrix of the user preference capture layer, b1, is the bias vector of the user preference capture layer, whose values ​​are determined by parameter learning, the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item, and ReLU is a nonlinear activation function;

[0107] Finally, after K repetitions of the above, a high-order embedding of the time interval perception of l items in the fixed-length training sequence is obtained;

[0108] Step S215: for the fixed-length training sequence, the time interval-aware high-order embeddings of l items in the sequence are grouped according to historical sessions, and a hybrid aggregator is used to generate a user's long-term preference, specifically including:

[0109] First, performing maximum pooling on the high-order embeddings of all items in several historical sessions of the user to generate embeddings of several historical sessions;

[0110] Then, average pooling is performed on the embeddings of the several historical sessions to generate a user's long-term preference;

[0111] Step S216: For a current session of the user, convert the current session into a session graph, which is defined as follows:

[0112] The session graph is a weighted directed graph, where each node in the graph represents an item in the current session, and each directed edge in the graph represents that the user interacts with the start item of the edge and then interacts with the end item of the edge in the current session. The weight of the edge is calculated by dividing the number of times the edge appears in the current session by the out-degree of the start item of the edge.

[0113] Step S217: for the session graph, a gated graph neural network is used to generate the embedding of each item in the session graph, and a local embedding and a global embedding of the current session are calculated, and then the two are concatenated and linearly transformed to generate the user's short-term preference, wherein the local embedding is the embedding of the last item in the current session, and the global embedding is calculated using soft attention, and the calculation process is as follows:

[0114] First, soft attention is used to calculate the weight coefficient between the embedding of the last item in the current session and the embedding of all items in the current session;

[0115] Then, the embeddings of all items in the current session are weighted and aggregated according to the calculated weight coefficients to generate a global embedding.

[0116] Step S22: constructing a user preference integration layer in the initial model of the session recommendation method ILSTP, and using the layer to generate an integrated user preference; the step S22 further includes:

[0117] Step S221: using an attention aggregator to generate an integrated user preference, specifically including:

[0118] First, a long-term preference coefficient is calculated for the user's long-term preference using an attention mechanism;

[0119] Then, a short-term preference coefficient is calculated for the user's short-term preference using an attention mechanism;

[0120] Finally, the long-term preference and short-term preference of the user are weighted and aggregated according to the calculated long-term preference coefficient and short-term preference coefficient to generate an integrated user preference.

[0121] Step S23: constructing a next item probability prediction layer in the initial model of the conversation recommendation method ILSTP, and using the layer to calculate the probability that each item in the conversation data set will become the next interactive item of the user; the step S23 further includes:

[0122] Step S231: performing vector dot product on the integrated user preferences and the initial embedding of each item in the conversation dataset to obtain prediction scores of these items;

[0123] Step S232: Use a softmax function to normalize the prediction score to generate the probability that each item in the conversation data set will become the user's next interaction item.

[0124] Step S3: adjusting the parameters of the initial model of the ILSTP using the training set and the validation set to obtain an optimized model of the ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model; the step S3 further includes:

[0125] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the adjusted model of the ILSTP on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0126] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the length l of the fixed-length training sequence, the number of layers K of the self-attention and feedforward neural networks, the learning rate η, and the regularization coefficient λ. The search range of these hyperparameters is as follows: the range of d is the set {10, 20, ..., 120}, the range of η is the set {0.005, 0.001, 0.01, 0.05, 0.1}, and the range of λ is the set {10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1}, the range of K is the set {1, 2, 3, 4}, and the range of l is the set {50, 60, ..., 200};

[0127] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the ILSTP parameter-adjusted model;

[0128] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0129] Step S32: Determine whether the recommendation accuracy of the ILSTP model after parameter adjustment reaches a predetermined threshold, where the recommendation accuracy is the recall rate, and the threshold value range is a closed interval [0.15, 0.25]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence, otherwise terminate the parameter adjustment process to obtain the optimized model of ILSTP;

[0130] Step S33: Use the test set and two performance evaluation indicators, namely, recall rate and average reciprocal rank, to evaluate the recommendation accuracy of the obtained ILSTP optimization model.

[0131] Step S4: Using the obtained ILSTP optimization model, provide the next recommendation for the user in the session data set or the actual session data.

[0132] The following is a specific example to further illustrate the specific implementation of the technical solution of the present invention, and through experiments, the technical solution of the present invention (i.e., the ILSTP method) is compared with other typical session recommendation methods based on deep neural networks, the HRNN model (see: Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi: Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks. RecSys 2017: 130-137. https: / / doi.org / 10.1145 / 3109859.3109896) and the A-PGNN model (see: Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, and Liang Wang: Personalized Graph Neural Networks with Attention Mechanism for Session-Aware Recommendation. IEEE Trans. Knowl. Data Eng. 15 October 2020 (Early Edition). Access).https: / / doi.org / 10.1109 / TKDE.2020.3031329) is used to compare the performance of recommendation accuracy, specifically using two performance evaluation indicators: recall rate (Recall@N) and mean reciprocal ranking (MRR@N).

[0133] The specific embodiment uses the MovieLens 20M dataset in the movie field (see: F. Maxwell Harper, Joseph A. Konstan: The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5(4): 19: 1-19: 19 (2016) https: / / doi.org / 10.1145 / 2827872). The MovieLens 20M dataset is an interactive dataset that contains about 20 million ratings (1 to 5) from 138,493 users on the MovieLens website for 27,278 movies. The method and steps for converting the interaction dataset MovieLens 20M into a session dataset are as follows: first, the interaction data in an interaction dataset are grouped by users, and then the interaction data of each user is processed as follows: the interaction data are sorted according to the temporal order of the interaction timestamps, and the sorted interaction data are divided into several sessions at a certain time interval, and several items with relatively close interaction timestamps in time form a session; for the session data of the user, a session composed of several items with the most recent and relatively close interaction timestamps in time is taken as a current session of the user, and several groups of items with earlier and relatively close interaction timestamps in time are taken as several historical sessions of the user; the session data of all users constitute a session dataset.

[0134] Table 1 gives a small part of the data in the MovieLens 20M dataset, and Table 2 gives a small part of the data in the conversation dataset converted from MovieLens20M.

[0135] Table 1. A small part of the data in the MovieLens 20M dataset

[0136] User ID Movie ID Users' Rating Timestamp (unit: seconds) 1 2 3.5 1112484727 1 29 3.5 1112485822 1 32 3.5 1112484819 ... ... ... 69496 21567 2.0 1164341241 69496 3364 3.5 1164339514 69496 7098 3.5 1164342574 69496 21893 0.5 1205466745 ... ... ... 138493 4816 3.0 1260209457 138493 9321 5.0 1258126944 138493 2890 2.5 1255811136

[0137] Table 2. A small portion of the conversational dataset converted from MovieLens 20M

[0138]

[0139]

[0140] Furthermore, in order to compare the performance of the technical solution of the present invention (i.e., the ILSTP model) with the HRNN model and the A-PGNN model in terms of recommendation accuracy through experiments using the above data sets, the inventors have managed to implement these three models, using Pycharm as the software development platform and Python as the programming language. The specific implementation techniques are as follows:

[0141] Implementation technology of HRNN model: The implementation tools are Python2.7 and theano 0.9.0. The code of this model is obtained directly from the GitHub library hgru4rec (see: https: / / github.com / mquad / hgru4rec)

[0142] Implementation technology of the A-PGNN model: The implementation tools are Python 3.6 and TensorFlow-gpu 1.10.0. The code of the model is obtained directly from the GitHub library A-PGNN (see: https: / / github.com / CRIPAC-DIG / A-PGNN).

[0143] Implementation technology of ILSTP model: The implementation tools are Python 3.6 and TensorFlow 1.15.0. The implementation technology is as follows: when building the user preference capture layer, download the code from GitHub (see: https: / / github.com / kang205 / SASRec) and make necessary modifications; when building the preference integration layer, download the code from GitHub (see: https: / / github.com / CRIPAC-DIG / SR-GNN) and make necessary modifications; when building the next probability prediction layer, use the nn class in the deep learning framework TensorFlow (see: https: / / github.com / tensorflow / tensorflow) and the dense class of the layer module to implement it.

[0144] The software operating environment of the experiment: (1) Hardware platform: Intel(R) Xeon(R) Silver 4116 CPU@2.10GHz×48, 2 processors, 12 cores each, 96GB memory, GPU model is GEFORCE RTX 2080Ti; (2) Operating system: Ubuntu 18.04.5 LTS; (3) Software platform: Pycharm.

[0145] The experimental process is as follows:

[0146] Experimental process of the HRNN model: (1) Set the values ​​of hyperparameters according to the recommendations given in the original publication of the HRNN model (see: Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi: Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks. RecSys 2017: 130-137. https: / / doi.org / 10.1145 / 3109859.3109896); (2) Calculate the recommendation accuracy of the HRNN model on the test set of the conversational dataset converted from MovieLens20M.

[0147] Experimental process of the A-PGNN model: (1) Set the values ​​of hyperparameters according to the recommendations given in the original publication of the A-PGNN model (see: Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, and Liang Wang: Personalized Graph Neural Networks with Attention Mechanism for Session-Aware Recommendation. IEEE Trans. Knowl. Data Eng. 15 October 2020 (Early Access). https: / / doi.org / 10.1109 / TKDE.2020.3031329); (2) Calculate the recommendation accuracy of the A-PGNN model on the test set of the conversational dataset converted from MovieLens 20M.

[0148] The experimental process of the ILSTP model is as follows: (1) generating initial embeddings for each item in a conversation dataset MovieLens 20M that includes several historical conversations and one current conversation of each user, and dividing the conversation dataset into a training set, a test set, and a validation set; (2) using the training set to construct an initial model of the next item recommendation method ILSTP that integrates long-term and short-term preferences; (3) using the training set and the validation set to adjust the parameters of the initial model of ILSTP to obtain an optimized model of ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model.

[0149] With respect to the specific embodiment, the specific implementation methods of the technical solution of the present invention are as follows:

[0150] like Figure 1 As shown, a next item recommendation method integrating a user's long-term preference and short-term preference includes the following steps S1 to S4:

[0151] Step S1: generating initial embeddings for each item in a session data set including several historical sessions of each user and one current session, and dividing the session data set into a training set, a test set, and a validation set; generating initial embeddings for the items in step S1 further includes:

[0152] Step S11: Use the item2vec embedding model to generate an initial embedding for each item in the conversation dataset, that is, represent an item as a d-dimensional vector, where the hyperparameter d∈{10, 20, ..., 120}; in this embodiment: the conversation dataset is a MovieLens The total number of sessions converted from the 20M data set is 392371; according to the common practice in the technical field, the session data set is first divided into a training set and a test set at a ratio of 8:2, and then 10% of the training set is used as a validation set; two points need to be further explained: (1) If the total number of sessions in some session data sets cannot be strictly divided into training set, test set and validation set according to the above ratio, then the number of sessions contained in the training set, test set and validation set respectively needs to be fine-tuned during the division process to maximize the closeness to the ratio. The session data set in this embodiment does encounter such a situation. Therefore, after the said fine-tuning in the experiment, the number of sessions contained in the training set, test set and validation set are 282507, 78474 and 31390 respectively; (2) According to the common practice in the technical field, a session data set can also be divided into a training set, a test set and a validation set in turn according to other division ratios, and the said division ratios include but are not limited to 8:1:1 and 7:1:2. The value of d is 100.

[0153] Step S2: Using the training set, construct an initial model of the next recommendation method ILSTP that integrates the user's long-term preference and short-term preference; Figure 2 and Figure 3 As shown, the step S2 further comprises:

[0154] Step S21: constructing a user preference capture layer in the initial model of the session recommendation method ILSTP, and using the layer to generate a user's long-term preference and a user's short-term preference; the step S21 further includes:

[0155] Step S211: for a user in the conversation data set, all items in several historical conversations of the user are connected into a training sequence, and the interaction timestamps of these items are connected into a time series accordingly, and then the training sequence and the time series are truncated or expanded into a fixed-length training sequence of length l and a fixed-length time series of length l, wherein the hyperparameter l∈{50, 60, ..., 200}, wherein the truncation occurs when the actual length of the sequence exceeds l, and the expansion occurs when the actual length of the sequence is less than l; in this embodiment: the value of l is 200;

[0156] Step S212: for the fixed-length training sequence, calculating the time interval Δt between any two items in the sequence, where the time interval is the absolute value of the difference between two interaction timestamps corresponding to the two items in the fixed-length time sequence;

[0157] Step S213: for the fixed-length training sequence, calculate the time interval embedding r between any two items in the sequence, the time interval embedding is calculated using a time interval embedding function, and the calculation method of the time interval embedding function is defined by the following formula:

[0158]

[0159] Where Δt is the time interval between any two items in a fixed-length training sequence, and σ is a sigmoid activation function;

[0160] Step S214: The layer number k of the K-layer self-attention and feedforward neural network is used as the loop variable of the counting control loop, whose initial value is 1, whose termination value is K, and whose incremental step is 1. The following steps (1) to (2) are repeated K times, where the hyperparameter K∈{1, 2, 3, 4}:

[0161] (1) For the fixed-length training sequence, calculate the k-th layer attention score between any two items v1 and v2 in the sequence And use a softmax function to Normalized to the k-th layer weight coefficient, as follows:

[0162] Based on the k-th layer embedding of items v1 and v2 and and the time interval between these two items is embedded in r 12 To calculate the attention score between these two items Among them, when k = 1, the first layer embedding of these two items and is the initial embedding of the item obtained in step S11, and the calculation method is defined by the following formula:

[0163]

[0164] Where T is the matrix transpose operation, W Q , is the weight matrix of the user preference capture layer, whose values ​​are determined by parameter learning, and the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item;

[0165] Get the above k-th layer attention score Then, a softmax function is used to convert Normalized to the k-th layer weight coefficient;

[0166] (2) For the fixed-length training sequence, calculate the k-th transformed embedding of l items in the sequence, as follows:

[0167] First, the k-th layer embeddings of all items in the sequence are weighted aggregated according to the k-th layer weight coefficient between any item in the sequence and all items in the sequence to generate the k-th layer aggregated embedding z of any item k ;

[0168] Then, a feed-forward neural network is used to k Transform to obtain the k-th transformed embedding f of the item k , the conversion method of the feedforward neural network is defined by the following formula:

[0169] f k =ReLU(z k W1+b1)W2+b2,

[0170] Where, W1, is the weight matrix of the user preference capture layer, b1, is the bias vector of the user preference capture layer, whose values ​​are determined by parameter learning, the hyperparameter d∈{10, 20, ..., 120} is the vector dimension of the initial embedding of the item, and ReLU is a nonlinear activation function;

[0171] Finally, after K repetitions of the above, a high-order embedding of the time interval perception of l items in the fixed-length training sequence is obtained; in this embodiment: the value of K is 3.

[0172] Step S215: for the fixed-length training sequence, the time interval-aware high-order embeddings of l items in the sequence are grouped according to historical sessions, and a hybrid aggregator is used to generate a user's long-term preference, specifically including:

[0173] First, performing maximum pooling on the high-order embeddings of all items in several historical sessions of the user to generate embeddings of several historical sessions;

[0174] Then, average pooling is performed on the embeddings of the several historical sessions to generate a user's long-term preference;

[0175] Step S216: For a current session of the user, convert the current session into a session graph, which is defined as follows:

[0176] The session graph is a weighted directed graph, where each node in the graph represents an item in the current session, and each directed edge in the graph represents that the user interacts with the start item of the edge and then interacts with the end item of the edge in the current session. The weight of the edge is calculated by dividing the number of times the edge appears in the current session by the out-degree of the start item of the edge.

[0177] Step S217: for the session graph, a gated graph neural network is used to generate the embedding of each item in the session graph, and a local embedding and a global embedding of the current session are calculated, and then the two are concatenated and linearly transformed to generate the user's short-term preference, wherein the local embedding is the embedding of the last item in the current session, and the global embedding is calculated using soft attention, and the calculation process is as follows:

[0178] First, soft attention is used to calculate the weight coefficient between the embedding of the last item in the current session and the embedding of all items in the current session;

[0179] Then, the embeddings of all items in the current session are weighted and aggregated according to the calculated weight coefficients to generate a global embedding.

[0180] Step S22: constructing a user preference integration layer in the initial model of the session recommendation method ILSTP, and using the layer to generate an integrated user preference; the step S22 further includes:

[0181] Step S221: using an attention aggregator to generate an integrated user preference, specifically including:

[0182] First, a long-term preference coefficient is calculated for the user's long-term preference using an attention mechanism;

[0183] Then, a short-term preference coefficient is calculated for the user's short-term preference using an attention mechanism;

[0184] Finally, the long-term preference and short-term preference of the user are weighted and aggregated according to the calculated long-term preference coefficient and short-term preference coefficient to generate an integrated user preference.

[0185] Step S23: constructing a next item probability prediction layer in the initial model of the conversation recommendation method ILSTP, and using the layer to calculate the probability that each item in the conversation data set will become the next interactive item of the user; the step S23 further includes:

[0186] Step S231: performing vector dot product on the integrated user preferences and the initial embedding of each item in the conversation dataset to obtain prediction scores of these items;

[0187] Step S232: Use a soffmax function to normalize the prediction score to generate the probability that each item in the conversation data set will become the user's next interaction item.

[0188] Step S3: adjusting the parameters of the initial model of the ILSTP using the training set and the validation set to obtain an optimized model of the ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model; the step S3 further includes:

[0189] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the adjusted model of the ILSTP on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0190] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the length l of the fixed-length training sequence, the number of layers K of the self-attention and feedforward neural networks, the learning rate η, and the regularization coefficient λ. The search range of these hyperparameters is as follows: the range of d is the set {10, 20, ..., 120}, the range of η is the set {0.005, 0.001, 0.01, 0.05, 0.1}, and the range of λ is the set {10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1}, the range of K is the set {1, 2, 3, 4}, and the range of l is the set {50, 60, ..., 200};

[0191] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the ILSTP parameter-adjusted model;

[0192] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0193] In this embodiment: the value of d is 100, the value of l is 200, the value of K is 3, the value of η is 0.001, and the value of λ is 10 -5 ;

[0194] Step S32: Determine whether the recommendation accuracy of the ILSTP model after parameter adjustment reaches a predetermined threshold, the recommendation accuracy is the recall rate, and the value range of the threshold is a closed interval [0.15, 0.25]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence, otherwise end the parameter adjustment process to obtain the optimized model of ILSTP; in this embodiment: the predetermined threshold of the recall rate is set to Recall@20=0.2;

[0195] Step S33: Use the test set and the two performance evaluation indicators of recall rate and average reciprocal ranking to evaluate the recommendation accuracy of the obtained ILSTP optimization model; in this embodiment: the recommendation accuracy results of the ILSTP optimization model are shown in Table 3, which also lists the recommendation accuracy results of the HRNN model and the A-PGNN model. The recommendation accuracy given in the table specifically has two evaluation indicators: recall rate (Recall@20) and average reciprocal ranking (MRR@20). The larger the indicator value, the better the performance.

[0196] Table 3. Next item recommendation accuracy results of three models on the conversational dataset converted from MovieLens 20M

[0197] Model Recall@20 MRR@20 ILSTP 0.203 0.110 HRNN 0.181 0.098 A-PGNN 0.199 0.107

[0198] Step S4: using the obtained ILSTP optimization model to provide the next recommendation to the users in the session data set or the actual session data; in this embodiment: using the obtained ILSTP optimization model to provide the next recommendation to the users in the test set.

[0199] The above experimental results show that compared with the HRNN model and the A-PGNN model, the recommendation accuracy of the next item recommendation method ILSTP that integrates the long-term preference and short-term preference of the user provided by the present invention is the highest among the three models.

[0200] In general, it can be understood from the above technical solutions of the present invention and its specific implementation methods that the beneficial technical effects of the present invention mainly include three aspects: (1) The present invention provides a next item recommendation method with high recommendation accuracy; (2) The next item recommendation method ILSTP of the present invention has higher recommendation accuracy than the typical deep neural network-based models HRNN and A-PGNN that capture users' long-term preferences and short-term preferences. The reason is that the technical solution of the present invention fully utilizes the time interval information between user-item interactions in the user's session data by organically combining time interval embedding functions, self-attention and hybrid aggregators to mine more accurate user preferences; (3) The present invention has broad application prospects in many fields such as music, movies, online news, and e-commerce.

[0201] The above is only part of the embodiments of the present invention. It should be noted that, except for the technical features described in the specification, all other technical features are known to those skilled in the art. For those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A next item recommendation method integrating a user's long-term preference and short-term preference, comprising the following steps: Step S1: Generate initial embeddings for each item in a session dataset containing several historical sessions and one current session of each user, and divide the session dataset into a training set, a test set, and a validation set; Step S2: Using the training set, an initial model of the next recommendation method ILSTP that integrates the user's long-term preference and short-term preference is constructed; specifically, as follows: Step S21: construct a user preference capture layer in the initial model of ILSTP, and use the layer to generate a user's long-term preference and a user's short-term preference; Step S22: constructing a user preference integration layer in the initial model of ILSTP, and using the layer to generate an integrated user preference; Step S23: constructing a next item probability prediction layer in the initial model of ILSTP, and using the layer to calculate the probability that each item in the conversation data set will become the next interaction item of the user; Step S3: adjusting the parameters of the initial model of the ILSTP using the training set and the validation set to obtain an optimized model of the ILSTP, and using the test set to evaluate the recommendation accuracy of the optimized model; Step S4: Using the obtained ILSTP optimization model, provide the next recommendation for the user in the session data set or the actual session data.

2. The method according to claim 1, characterized in that The initial embedding of the generated project in step S1 further comprises the following steps: Step S11: Use the item2vec embedding model to generate an initial embedding of each item in the conversation dataset, that is, represent an item as a d-dimensional vector, where the hyperparameter d∈{10, 20,…, 120}.

3. The method according to claim 1 or 2, characterized in that: The step S21 further comprises the following steps: Step S211: for a user in the conversation data set, all items in several historical conversations of the user are connected into a training sequence, and the interaction timestamps of the items are connected into a time series accordingly, and then the training sequence and the time series are truncated or expanded into a fixed-length training sequence of length l and a fixed-length time series of length l, where the hyperparameter l∈{50, 60, ..., 200}, where the truncation occurs when the actual length of the sequence exceeds l, and the expansion occurs when the actual length of the sequence is less than l; Step S212: for the fixed-length training sequence, calculating the time interval Δt between any two items in the sequence, where the time interval is the absolute value of the difference between two interaction timestamps corresponding to the two items in the fixed-length time sequence; Step S213: for the fixed-length training sequence, calculate the time interval embedding r between any two items in the sequence, the time interval embedding is calculated using a time interval embedding function, and the calculation method of the time interval embedding function is defined by the following formula: Where Δt is the time interval between any two items in a fixed-length training sequence, and σ is a sigmoid activation function; Step S214: The layer number k of the K-layer self-attention and feedforward neural network is used as the loop variable of the counting control loop, whose initial value is 1, whose termination value is K, and whose incremental step is 1. The following steps (1) to (2) are repeated K times, where the hyperparameter K∈{1, 2, 3, 4}: (1) For the fixed-length training sequence, calculate the k-th layer attention score between any two items v1 and v2 in the sequence And use a softmax function to Normalized to the k-th layer weight coefficient, as follows: Based on the k-th layer embedding of items v1 and v2 and and the time interval between these two items is embedded in r 12 To calculate the attention score between these two items Among them, when k = 1, the first layer embedding of these two items and is the initial embedding of the item obtained in step S11, and the calculation method is defined by the following formula: Where T is the matrix transpose operation, W Q , is the weight matrix of the user preference capture layer, whose values ​​are determined by parameter learning, and the hyperparameter d∈{10, 20,…, 120} is the vector dimension of the initial embedding of the item; Get the above k-th layer attention score Then, a softmax function is used to convert Normalized to the k-th layer weight coefficient; (2) For the fixed-length training sequence, calculate the k-th transformed embedding of l items in the sequence, as follows: First, the k-th layer embeddings of all items in the sequence are weighted aggregated according to the k-th layer weight coefficient between any item in the sequence and all items in the sequence to generate the k-th layer aggregated embedding z of any item k ; Then, a feed-forward neural network is used to k Transform to obtain the k-th transformed embedding f of the item k , the conversion method of the feedforward neural network is defined by the following formula: <h2 style=";text-align:left;direction:ltr">f<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> =ReLU(z<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> W1+b1)W2+b2, Where, W1, is the weight matrix of the user preference capture layer, b1, is the bias vector of the user preference capture layer, whose values ​​are determined by parameter learning, the hyperparameter d∈{10, 20,…, 120} is the vector dimension of the initial embedding of the item, and ReLU is a nonlinear activation function; Finally, after K repetitions of the above, a high-order embedding of the time interval perception of l items in the fixed-length training sequence is obtained; Step S215: for the fixed-length training sequence, the time interval-aware high-order embeddings of l items in the sequence are grouped according to historical sessions, and a hybrid aggregator is used to generate a user's long-term preference, specifically including: First, performing maximum pooling on the high-order embeddings of all items in several historical sessions of the user to generate embeddings of several historical sessions; Then, average pooling is performed on the embeddings of the several historical sessions to generate a user's long-term preference; Step S216: For a current session of the user, convert the current session into a session graph, which is defined as follows: The session graph is a weighted directed graph, where each node in the graph represents an item in the current session, and each directed edge in the graph represents that the user interacts with the start item of the edge and then interacts with the end item of the edge in the current session. The weight of the edge is calculated by dividing the number of times the edge appears in the current session by the out-degree of the start item of the edge. Step S217: for the session graph, a gated graph neural network is used to generate the embedding of each item in the session graph, and a local embedding and a global embedding of the current session are calculated, and then the two are concatenated and linearly transformed to generate the user's short-term preference, wherein the local embedding is the embedding of the last item in the current session, and the global embedding is calculated using soft attention, and the calculation process is as follows: First, soft attention is used to calculate the weight coefficient between the embedding of the last item in the current session and the embedding of all items in the current session; Then, the embeddings of all items in the current session are weighted and aggregated according to the calculated weight coefficients to generate a global embedding.

4. The method according to claim 3, characterized in that The step S22 further comprises the following steps: Step S221: using an attention aggregator to generate an integrated user preference, specifically including: First, a long-term preference coefficient is calculated for the user's long-term preference using an attention mechanism; Then, a short-term preference coefficient is calculated for the user's short-term preference using an attention mechanism; Finally, the long-term preference and short-term preference of the user are weighted and aggregated according to the calculated long-term preference coefficient and short-term preference coefficient to generate an integrated user preference.

5. The method according to claim 4, characterized in that The step S23 further comprises the following steps: Step S231: performing vector dot product on the integrated user preferences and the initial embedding of each item in the conversation dataset to obtain prediction scores of these items; Step S232: Use a softmax function to normalize the prediction score to generate the probability that each item in the conversation data set will become the user's next interaction item.

6. The method according to claim 5, characterized in that The step S3 further comprises the following steps: Step S31: Using the Adam optimizer to learn the parameters of the initial model or the adjusted model of the ILSTP on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including: First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the length l of the fixed-length training sequence, the number of layers K of the self-attention and feedforward neural networks, the learning rate η, and the regularization coefficient λ. The search range of these hyperparameters is as follows: the range of d is the set {10, 20, ..., 120}, the range of η is the set {0.005, 0.001, 0.01, 0.05, 0.1}, and the range of λ is the set {10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1 }, the range of K is the set {1, 2, 3, 4}, and the range of l is the set {50, 60,…, 200}; Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the ILSTP parameter-adjusted model; Finally, the recommendation accuracy of the model is calculated on the validation set; Step S32: Determine whether the recommendation accuracy of the ILSTP model after parameter adjustment reaches a predetermined threshold, where the recommendation accuracy is the recall rate, and the threshold value range is a closed interval [0.15, 0.25]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence, otherwise terminate the parameter adjustment process to obtain the optimized model of ILSTP; Step S33: Use the test set and two performance evaluation indicators, namely, recall rate and average reciprocal rank, to evaluate the recommendation accuracy of the obtained ILSTP optimization model.

Images