Sequence recommendation method based on global information and hybrid time query attention mechanism

Abstract

This invention discloses a sequence recommendation method based on global information and a hybrid temporal query attention mechanism, comprising: S1, capturing global information between items in a user interaction sequence using a global item transition graph to obtain graph embeddings; S2, inputting the user's interaction sequence into an embedding layer to generate sequence embeddings of the mean and covariance of items in the user interaction sequence; S3, inputting the sequence embeddings generated in S2 into a hybrid temporal query self-attention module to output new sequence embeddings of the mean and covariance of the user interaction sequence; S4, introducing the output of S3 into a feedforward network layer to obtain updated sequence embeddings; S5, defining the sequence embeddings updated by the feedforward network layer in step S4 as short-term sequence embeddings and long-term sequence embeddings; S6, generating final embeddings based on short-term and long-term sequence embeddings; and S7, performing recommendation tasks based on the final embeddings output in S6. This invention can improve the performance and accuracy of recommendation systems.

Description

Technical Field

[0001] This invention relates to the field of recommender system technology, specifically a sequence recommendation method based on global information and a hybrid temporal query attention mechanism. Background Technology

[0002] With the explosive growth of online information, recommender systems have played a crucial role in alleviating information overload. Recommender systems can select target data of interest to users from massive amounts of data, possessing significant application value. Traditional recommendation methods include collaborative filtering, content-based recommendation, and tag-based recommendation, which statically capture user preferences and model interactions between users and items. However, in real life, user behavior occurs sequentially, with strong connections between actions, and user preferences are dynamic and change over time. Therefore, methods capable of modeling dynamic user preferences are needed. Sequence recommendation primarily recommends items that users may be interested in by modeling user-item interactions within sequences (e.g., the sequential dependencies between viewing or purchasing items on an online shopping platform).

[0003] Currently, sequence recommendation methods can be broadly categorized into three main types: traditional sequence models, latent representation models, and deep neural network models. Traditional sequence models include sequence pattern mining and Markov chain models, which utilize natural strength to model sequential dependencies between user-item interactions in a sequence. Sequence pattern mining can discover implicit correlations and scalability, but it is affected by data sparsity and implicit patterns; Markov chain models are simple, intuitive, and computationally efficient, but rely on assumptions and lack global information. The choice of model suitable for a specific application scenario depends on data attributes, modeling requirements, and performance requirements. Latent representation models include decomposers and embeddings, which first learn latent representations for each user or item, and then use the learned representations to predict subsequent user-item interactions. Their advantages include the ability to extract meaningful features, reduce the curse of dimensionality, and improve model generalization ability. However, latent representation models also have some drawbacks and problems, such as difficulty in interpreting the learned representations, potential information loss, and insufficient generalization ability. With the development of deep learning, deep neural networks have been integrated into sequence recommendation, and recurrent neural networks (RNNs) and their variants have been applied to user behavior sequences. Besides RNNs, other deep learning models such as Transformers have also been adopted for sequential recommendation tasks and have achieved good performance. Transformer-based sequential recommendation excels at capturing the dynamic characteristics of user interests by modeling complex item-item transition relationships within sequences. For example, SASRec pioneered the use of Transformers for sequential recommendation through a scaled dot product attention mechanism, significantly improving the understanding of how items are interconnected over time. Later, BERT4Rec proposed end-to-end historical sequence recommendation using a bidirectional attention mechanism. TiSASRec aims to address the problem of modeling temporal information to more accurately capture the time interval relationships between items.

[0004] Despite their success in sequence recommendation, some limitations remain:

[0005] First, Transformer-based sequence recommendation has been widely used in recommender systems due to its excellent ability to handle sequence data. These models capture long-distance dependencies in sequences through self-attention mechanisms and effectively understand the contextual information of user behavior sequences, thereby predicting users' interest in unknown items. However, they have certain limitations in handling spatial information between items, especially the similarity or relevance between items. Although some models can indirectly capture the relevance between items through user-item interactions, this relevance is usually implicit. The lack of explicit modeling of direct associations between items prevents recommender systems from fully utilizing existing knowledge of inter-item relationships, thus affecting recommendation performance.

[0006] Secondly, typical self-attention models essentially restrict themselves to a first-order Markov chain model that enhances attention, using only the most recently embedded items as queries within the self-attention module. This design limits the model's ability to leverage broader user behavior patterns for recommendations, as it primarily focuses on direct item-to-item transitions rather than capturing collaborative behaviors or long-term interest patterns among users. Consequently, its recommendation performance is poor in scenarios requiring the understanding and utilization of collaborative signals between users. Summary of the Invention

[0007] Purpose of the invention: To address the limitations of existing recommender systems in processing user behavior sequences, particularly in capturing direct relationships between items and understanding user collaboration, this invention proposes a sequence recommendation method based on global information and a hybrid temporal query attention mechanism. This method can fully utilize global information and user collaboration information to improve the performance and accuracy of the recommender system.

[0008] Technical solution: A sequence recommendation method based on global information and a hybrid temporal query attention mechanism, comprising the following steps:

[0009] S1. Use a global item transformation graph to capture global information between items in the user interaction sequence, define a rule-based global item graph, use the global item graph learned by singular value decomposition to obtain an augmented graph, and obtain the graph embedding through graph encoding.

[0010] S2. Input the user's interaction sequence into the embedding layer, represent the items in the user's interaction sequence using a multidimensional elliptic Gaussian distribution, and generate sequence embeddings of the mean and covariance of the items in the user's interaction sequence.

[0011] S3. Embed the sequence generated in S2 and input it into the hybrid temporal query self-attention module. Use a sliding window and calculate the query vector using the time decay weighted average method. Add the position information as a bias to the calculation of the attention weight to obtain the output of the hybrid temporal query self-attention module. The output result is a new sequence embedding with the mean and covariance of the user interaction sequence.

[0012] S4. Embed the new sequence output from S3 and introduce it into the feedforward network layer. By adding nonlinearity, perform residual connection, layer normalization and random loss operation to obtain the updated sequence embedding.

[0013] S5. The sequence embedding updated by the feedforward network layer in step S4 is defined as short-term sequence embedding and long-term sequence embedding according to the length of the time window.

[0014] S6. A new short-term sequence embedding is obtained by fusing the short-term embedding obtained in S5 and the graph embedding obtained in S1, and the new short-term embedding is combined with the long-term embedding obtained in S5 to generate the final embedding.

[0015] S7. Based on the final embedding output from S6, the Wasserstein distance is used to calculate the distance between the distributions of two items in the user interaction sequence, which is used as the ranking score to generate a recommendation list. Bayesian personalized ranking loss is used as the basic loss function, and a regularization term that considers the distance between positive and negative sampled items and global information encoding loss are introduced to optimize the basic loss function for sequence recommendation.

[0016] Furthermore, the specific steps of S1 are as follows:

[0017] S1.1 Define the user's interaction sequence Item v in the sequence i and Project v i+k The edge update weight between them is set to Where v i v represents the i-th item in the user interaction sequence. i+k This represents the score of the (i+k)th item in the user interaction sequence. Represents the target node v i Its k-jump neighbor v i+k The importance of;

[0018] S1.2 Repeat the S1.1 operation for all user interaction sequences to construct the global project graph G, and then perform operations on the two nodes v. i and v i+k Normalize the edge weights between them:

[0019]

[0020] in, It is an adjacency matrix. G is the normalized adjacency matrix, and deg(·) is the degree of a node in G;

[0021] S1.3. Perform SVD processing on G to obtain the enhanced map. The expression for SVD is:

[0022]

[0023] Among them, U q and V q For a standard orthogonal matrix, S q It is a diagonal matrix; This represents the normalized adjacency matrix;

[0024] S1.4. Capture global information in the form of a graph, and use LightGCN's graph encoding method to obtain the graph representation. The resulting graph embedding expression is:

[0025]

[0026] Among them, e c For node embedding in layer c, where c represents the number of layers in the graph convolution and β represents the adjacency matrix. This represents the normalized adjacency matrix.

[0027] Furthermore, the specific steps of S2 are as follows:

[0028] For all projects, a multidimensional elliptic Gaussian distribution is used to describe them. First, a mean embedding table M is defined. μ ∈R |V|×d and a covariance embedding table M ∑ ∈R |V|×d Where d represents the number of latent dimensions and V represents the number of items; then the sequence embeddings of the mean and covariance of items in the user interaction sequence are as follows:

[0029]

[0030]

[0031] in, Sequence embeddings representing the mean of the items. Sequence embeddings representing the covariance of the project. This represents the mean embedding of the nth item in the user interaction sequence. This represents the covariance embedding of the nth item in a user interaction sequence. This represents the nth item in the mean embedding table. This represents the nth item in the covariance embedding table.

[0032] Furthermore, the specific steps of S3 are as follows:

[0033] S3.1 Obtain the time query embedding using the following formula.

[0034]

[0035] in, It is a normalization factor that ensures the sum of the weights is 1; e -d·l is the exponentially decaying weight of the l-th element, and L is a hyperparameter used to control the scope of the attention query; This represents the embedding at time t-L+l;

[0036] S3.2. The Wasserstein distance is used to measure the distance between two random embeddings;

[0037] Given items s in a user's interaction sequence i and s j The corresponding random embeddings are respectively and For project s i , and These are the mean embedding and covariance embedding of the i-th item in the user interaction sequence, respectively. and All parameters are learnable, ELU represents the activation function, and diag represents the extraction of the diagonal matrix. Attention weights are defined by measuring the negative 2-Wasserstein distance W2(·,·), and positional information is added to the calculation of attention weights as a bias to obtain attention weight A. ij The calculation formula is as follows:

[0038] A ij =-W2(s i ,s j )+m·[-(p-1),...,-2,-1,0]

[0039] in, For the second-order Wasserstein distance, A ij For attention weights, m·[-(p-1),...,-2,-1,0] is a relative distance matrix, where p represents the position index and m is an adjustment parameter that can be set according to the number of attention heads;

[0040] S3.3 Each item is represented by a random embedding containing the mean and covariance, and information is aggregated using the properties of linear combinations of Gaussian distributions, as shown in the following formula:

[0041]

[0042]

[0043] in, These are the normalized attention weight values. and The value vectors representing the mean and covariance, respectively, are obtained by performing a linear transformation on the initial sequence embedding. This represents the new sequence mean embedding after aggregating information. This represents the new sequence covariance embedding after aggregating information, where j takes values ​​from 1 to n.

[0044] Furthermore, the specific steps of S4 are as follows:

[0045] S4.1 Embed the new sequence output from S3 and introduce it into the feedforward network layer to capture more complex relationships. The feedforward network layer first passes through a multilayer perceptron (MLP) and then through a linear layer, as shown in the following formula:

[0046]

[0047]

[0048] in, and These are the mean output and covariance output of the feedforward network layer, respectively. All of these are learnable parameters;

[0049] S4.2. Using residual connections, layer normalization, and random loss operations, update the feedforward network layer output as follows:

[0050]

[0051]

[0052] in, This represents the updated sequence mean embedding. This represents the updated sequence covariance embedding.

[0053] Furthermore, in S5, the new sequence output from the feedforward network layer in step S4 is embedded, and the method for defining short-term sequence embedding and long-term sequence embedding according to the length L of the time window is as follows:

[0054] When L=1, the output of the feedforward network layer is a short-term sequence embedding; when L=5, the output of the feedforward network layer is a long-term sequence embedding.

[0055] Furthermore, the specific steps of S6 are as follows:

[0056] S6.1 The formula for obtaining the new short-term sequence embedding is as follows:

[0057]

[0058] in, middle This indicates that step five yields the short-term sequence embeddings. This represents the graph embedding obtained from step one, and the updated result obtained by applying a FuseLayer layer to these two parts.

[0059] S6.2. Using the integrated query embedding method, the new short-term sequence embeddings and long-term sequence embeddings obtained in step 6.1 are combined to generate the final embedding; the formula is as follows:

[0060]

[0061]

[0062] in Embedding for new short-term sequences; For long-term sequence embeddings; λ is a learnable hyperparameter.

[0063] Furthermore, in S7,

[0064] (1) Calculate the Wasserstein distance between the distributions of two items to predict the ranking score for item s. i The next project r The predicted score is calculated using the following formula:

[0065]

[0066] (2) Bayesian personalized ranking is used as the basic loss function, and a regularization term considering the distance between positive and negative sampling items and global information encoding loss are introduced to optimize the basic loss function;

[0067] Regularization term l pvn The formula is as follows:

[0068]

[0069] Where, [x] + =max(x,0), k + It is the next item the user visits, i.e., the main item, k - These are negative items randomly sampled from items in the user interaction sequence that have never been interacted with; s t This represents the t-th item in the user interaction sequence; This represents the distance between the t-th item and the positive item in the user interaction sequence. The distance between positive and negative items is represented by the following formula for global information encoding loss:

[0070]

[0071] in, It is the embedding obtained by graph encoding from the original global graph G. It is an enhanced graph The embedding obtained after graph encoding, where N represents the number of samples;

[0072] The final loss function is:

[0073]

[0074] Among them, S u Let σ represent the user's interaction sequence, θ represent the logistic function, θ represent all learnable parameters, ω represent a hyperparameter, and α1 and α2 represent two hyperparameters used to control two constraint terms.

[0075] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0076] This invention proposes a sequence recommendation method based on global information and a hybrid temporal query attention mechanism, which addresses the limitations of existing recommender systems in capturing dynamic user preferences and modeling relationships between items in a sequence. Compared to traditional recommendation methods, this invention, by capturing global information between items in a sequence and combining it with a hybrid temporal attention mechanism, can understand user collaboration information, thereby more accurately predicting user interests and behaviors. This method not only improves the performance and accuracy of recommender systems but also has strong versatility and applicability, and can be widely applied to various types of recommender systems, providing users with a personalized recommendation experience. In conclusion, this invention has significant technical value and application prospects in the field of recommender systems. Attached Figure Description

[0077] Figure 1 This is a flowchart of the present invention;

[0078] Figure 2 This is a framework diagram of an example of the present invention, HWSA-SF. Detailed Implementation

[0079] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0080] like Figure 1 As shown, this invention provides a sequence recommendation method based on global information and a hybrid temporal query attention mechanism, comprising the following steps:

[0081] S1. Use a global item transformation graph to capture global information between items in the user interaction sequence. Define a rule-based global item graph, use the global item graph learned by singular value decomposition (SVD) to obtain an augmented graph, and obtain the graph embedding through graph encoding.

[0082] Traditional sequence modeling methods primarily focus on modeling sequence transition patterns within a single sequence, neglecting the use of global information between sequences, which may result in poor user representation quality. To overcome this problem, this invention uses a global item transition graph to capture global information between sequences.

[0083] The specific steps of S1 are as follows:

[0084] S1.1 Define the user's interaction sequence Item v in the sequence i and Project v i+k The edge update weight between them is set to Where v i v represents the i-th item in the user interaction sequence. i+k This represents the score of the (i+k)th item in the user interaction sequence. Represents the target node v i Its k-jump neighbor v i+k The importance of.

[0085] S1.2 Repeat the S1.1 operation for all user interaction sequences to construct the global project graph G, and then perform operations on the two nodes v. i and v i+k Normalize the edge weights between them:

[0086]

[0087] in, It is an adjacency matrix. Let G be the normalized adjacency matrix, and deg(·) be the degree of a node in G.

[0088] S1.3. Perform SVD processing on G to obtain the enhanced map. The expression for SVD is:

[0089]

[0090] Among them, U q and V q For a standard orthogonal matrix, S q It is a diagonal matrix. This represents the normalized adjacency matrix.

[0091] S1.4. Capture global information in the form of a graph, and use LightGCN's graph encoding method to obtain the graph representation. The resulting graph embedding expression is:

[0092]

[0093] Among them, e c For node embedding in layer c, where c represents the number of layers in the graph convolution and β represents the adjacency matrix. This represents the normalized adjacency matrix.

[0094] S2. Input the user's interaction sequence into the embedding layer, represent the items in the user interaction sequence with a multidimensional elliptic Gaussian distribution, and generate sequence embeddings of the mean and covariance of the items in the user interaction sequence to capture the uncertainty of the items.

[0095] This invention employs a unique method to represent items, incorporating the concept of probability distributions rather than relying on traditional deterministic vector representations. The essence of this method lies in utilizing a multidimensional elliptic Gaussian distribution to describe items, effectively capturing potential uncertainties. Specifically, the elliptic Gaussian distribution is governed by an average vector and a covariance vector, where the covariance vector introduces the potential uncertainty of the item.

[0096] The specific steps of S2 are as follows:

[0097] For all projects, a multidimensional elliptic Gaussian distribution is used to describe them. First, a mean embedding table M is defined. μ ∈R |V|×d and a covariance embedding table M ∑ ∈R |V|×d Where d represents the number of latent dimensions and V represents the number of items. Since the mean and covariance can capture different information about the data, the sequence embeddings of the mean and covariance of items in the user interaction sequence were obtained, as follows:

[0098]

[0099]

[0100] in, Sequence embeddings representing the mean of the items. Sequence embeddings representing the covariance of the project. This represents the mean embedding of the nth item in the user interaction sequence. This represents the covariance embedding of the nth item in a user interaction sequence. This represents the nth item in the mean embedding table. This represents the nth item in the covariance embedding table.

[0101] S3. Embed the sequence generated in S2 and input it into the hybrid temporal query self-attention module. Use a sliding window and calculate the query vector using the time decay weighted average method. Add the position information as a bias to the calculation of the attention weight to obtain the output of the hybrid temporal query self-attention module. The output result is a new sequence embedding with the mean and covariance of the user interaction sequence.

[0102] The time-query self-attention mechanism is a variant of self-attention that uses embedding as its query mechanism within the L most recent timestamps.

[0103] The steps for S3 are as follows:

[0104] S3.1 Obtain the time query embedding using the following formula.

[0105]

[0106] in, It is a normalization factor that ensures the sum of the weights is 1; e -d·l is the exponentially decaying weight of the l-th element, and L is a hyperparameter used to control the scope of the attention query. This represents the embedding at time t-L+l. Information from the past L time points is integrated and weighted by time decay so that the most recent time points contribute more to the results.

[0107] S3.2 Traditional self-attention weight calculation cannot measure the distance between two distributions, nor does it satisfy the triangle inequality. Therefore, this invention uses Wasserstein distance to measure the distance between two random embeddings. Given two user interaction sequences of items s i and s j The corresponding random embeddings are respectively and Attention weights are defined by measuring the negative 2-Wasserstein distance W2(·,·), and positional information is added to the attention weights A as a bias. ij The calculation formula is as follows:

[0108] A ij =-W2(s i ,s j )+m·[-(p-1),...,-2,-1,0]

[0109] in, For the second-order Wasserstein distance, A ij For attention weights, m·[-(p-1),...,-2,-1,0] is a relative distance matrix, where p represents the position index; the adjustment parameter m can be set according to the number of attention heads. For a model with 4 attention heads, the sequence of m values ​​is a geometric sequence: 2 -2 ,2 -4 ,2 -6 ,2 -8 Generally, for an n-head attention mechanism, the value of m is a value derived from... The initial geometric sequence, and the ratio is also...

[0110] S3.3 Since each item is represented by a random embedding containing the mean and covariance, it is necessary to summarize the mean and covariance separately. The information is aggregated using the property of linear combinations of Gaussian distributions, as shown in the following formula:

[0111]

[0112]

[0113] in, The normalized value. and The value vectors representing the mean and covariance, respectively, are obtained by performing a linear transformation on the initial sequence embedding. This represents the new sequence mean embedding after aggregating information. This represents the new sequence covariance embedding after aggregating information, where j takes values ​​from 1 to n.

[0114] S4. Embed the new sequence output from S3 and introduce it into the feedforward network layer. By adding nonlinearity, perform residual connection, layer normalization and random loss operation to obtain the updated sequence embedding.

[0115] The self-attention mechanism aggregates the embeddings of historical items through adaptive weights, enhancing the model's expressive power. However, its inherent linearity limits its ability to capture complex relationships between items. Therefore, this invention introduces a feedforward network layer.

[0116] The specific steps for S4 are as follows:

[0117] S4.1 Embed the new sequence output from S3 and introduce it into the feedforward network layer to capture more complex relationships. The feedforward network layer first passes through a multilayer perceptron (MLP) and then through a linear layer, as shown in the following formula:

[0118]

[0119]

[0120] in, and These are the mean output and covariance output of the feedforward network layer, respectively. All of these are learnable parameters;

[0121] S4.2 In addition, residual connections, layer normalization, and random loss operations are used to update the feedforward network layer output as follows:

[0122]

[0123]

[0124] in, This represents the updated sequence mean embedding. This represents the updated sequence covariance embedding. The ELU activation function and an increment operation are used to ensure the positive definiteness of the covariance.

[0125] S5. The sequence embeddings updated by the feedforward network layer in step S4 are defined as short-term sequence embeddings and long-term sequence embeddings based on the length of the time window; specifically,

[0126] This is the output of the feedforward network layer when L=1, i.e., the short-term sequence embedding. This represents the output of the feedforward network layer when L=5, i.e., the long-term sequence embedding. The short-term sequence embedding with L=1 and the long-term sequence embedding with L=5 are the best results obtained through multiple experiments.

[0127] S6. A new short-term sequence embedding is obtained by fusing the short-term embedding obtained in S5 and the graph embedding obtained in S1. The new short-term embedding is then combined with the long-term embedding obtained in S5 to generate the final embedding, in order to balance the variance and bias in capturing user preferences.

[0128] The specific steps are as follows:

[0129] Step 6.1, obtain the new short-term sequence embedding, using the following formula:

[0130]

[0131] in, This represents the mean output of step 4.2 when L=1. This represents the graph embedding obtained from step 1.4. The updated graph is obtained by applying a FuseLayer layer to these two parts. The FuseLayer layer concatenates the two inputs together through a linear transformation and maps them to the same dimension as the original mean sequence output.

[0132] Step 6.2: Using the integrated query embedding method, the new short-term sequence embedding and long-term sequence embedding obtained in Step 6.1 are combined to generate the final embedding;

[0133] For step S3, choosing the correct L value is crucial. A larger L value makes the model rely on longer historical information to capture the evolution of user interests, which helps capture collaborative signals but may lead to accumulated bias in the model over time. Conversely, a smaller L value makes the model rely more on recent interaction information to represent user interests; while this reduces the risk of introducing bias, it also increases variance. To strike a balance between bias and variance, a suitable L value needs to be carefully chosen.

[0134] Therefore, this invention designs an integrated query embedding method that integrates short-term sequence embedding and long-term sequence embedding to generate the final embedding, as shown in the following formula:

[0135]

[0136]

[0137] in, Embed the new short sequence obtained in step 6.1; For long-term sequence embeddings; λ is a learnable hyperparameter.

[0138] S7. Based on the final embedding output from S6, the Wasserstein distance is used to calculate the distance between the distributions of two items in the user interaction sequence, which is used as the ranking score to generate a recommendation list. Bayesian personalized ranking loss is used as the basic loss function, and regularization terms that consider the distance between positive and negative sampled items and global information encoding loss are introduced to optimize the basic loss function for sequence recommendation.

[0139] Based on the final embedding output by S6, the next possible interaction item in the user interaction sequence is predicted. Since the distributions of two items are given, the Wasserstein distance between the distributions of the two items needs to be calculated to predict the ranking score. Specifically, for item s... i The next project r The predicted score is calculated using the following formula:

[0140]

[0141] This invention uses Bayesian personalized ranking as the basic loss function and introduces a regularization term that considers the distance between positive and negative sampled items and a global information encoding loss to optimize the basic loss function.

[0142] Regularization term l pvn The formula is as follows:

[0143]

[0144] Where, [x] + =max(x,0), k + It is the next item the user visits, i.e., the main item, k - These are negative items randomly sampled from items in the user interaction sequence that have never been interacted with; s t This represents the t-th item in the user interaction sequence; This represents the distance between the t-th item and the positive item in the user interaction sequence. It represents the distance between positive and negative items.

[0145] The formula for the global information encoding loss is as follows:

[0146]

[0147] in It is the embedding obtained by graph encoding from the original global graph G. It is an enhanced graph The embedding obtained after graph encoding, where N represents the number of samples.

[0148] The final loss function is:

[0149]

[0150] Among them, S u Let σ represent the user's interaction sequence, θ represent the logistic function, θ represent all learnable parameters, ω represent a hyperparameter, and α1 and α2 represent two hyperparameters used to control two constraint terms.

[0151] This embodiment uses two public datasets from Amazon for training. These datasets contain over 1.2 million users and 63,000 items, demonstrating the high sparsity and diverse rating categories characteristic of Amazon datasets. This embodiment uses the PyTorch library in Python to experiment with the HWSA-SF model, which is constructed using the sequence recommendation method based on global information and a hybrid temporal query attention mechanism proposed in this invention.

[0152] This embodiment selects eight methods for comparison to evaluate the effectiveness and superiority of the proposed HWSA-SF in the field of recommender systems. These comparison methods are divided into three categories: (1) static recommendation methods, such as BPRMF and LightGCN; (2) Transformer-based sequence methods, including BERT4Rec, SASRec and STOSA; and (3) graph-based sequence methods, such as SR-GNN and GCE-GNN.

[0153] Recall@5 and NDCG@5 were used as quantitative indicators to evaluate various models, and the quantitative evaluation results are shown in Table 1.

[0154] Table 1 Comparison of experimental results

[0155]

[0156] The following conclusions can be drawn from Table 1:

[0157] (1) The HWSA-SF proposed in this invention consistently outperforms all baselines on the dataset. The results demonstrate the effectiveness of HWSA-SF in sequence recommendation tasks.

[0158] (2) Clearly, static recommendation methods perform the worst when comparing the three different types of recommendation methods, indicating that temporal information is a very important factor in sequence recommendation. Among graph-based sequence methods, GCE-GNN outperforms SR-GNN, which can be attributed to the global collaborative information it introduces. Furthermore, Transformer-based sequence methods perform best, with STOSA showing the best performance, followed by SASRec. STOSA outperforms SASRec on all datasets, demonstrating the importance of uncertainty information modeling and also validating the effectiveness of the Transformer framework.

[0159] (3) This invention designs a global learner that uses singular value decomposition to integrate global collaborative information into the item representation, which can more accurately capture the correlation between different items and user preferences, thereby enhancing the robustness of the recommendation system. In addition, a hybrid temporal query self-attention module is introduced, which achieves a balance between capturing variance and bias in user preferences, and helps to improve the understanding of user interests and modeling capabilities.

[0160] In summary, the recommendation performance of the model proposed in this invention is superior to other baseline models, demonstrating the effectiveness and superiority of this solution.

[0161] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A sequence recommendation method based on global information and hybrid time query attention mechanism, characterized by, S1, using a global item conversion graph to capture the global information between items in the user interaction sequence, defining a rule-based global item graph, using singular value decomposition learning to obtain an enhanced graph, and obtaining graph embedding through graph encoding; S2, input the user's interaction sequence into the embedding layer, represent the items in the user interaction sequence with a multi-dimensional elliptical Gaussian distribution, generate the sequence embedding of the mean and covariance of the items in the user interaction sequence; S3, input the sequence embedding generated in S2 into the hybrid time query self-attention module, use a sliding window, calculate the query vector by time decay weighted average method, and add the position information to the calculation of the attention weight in the form of bias, obtain the output of the hybrid time query self-attention, and the output result is the new sequence embedding of the mean and covariance of the user interaction sequence; S4, introduce the feedforward network layer to the new sequence embedding output in S3, increase nonlinearity, and then perform residual connection, layer normalization and random loss operation to obtain the updated sequence embedding; S5, according to the length of the time window, define the sequence embedding as short-term sequence embedding and long-term sequence embedding; S6, fuse the short-term sequence embedding obtained in S5 and the graph embedding obtained in S1 to obtain a new short-term sequence embedding, and combine the new short-term sequence embedding with the long-term sequence embedding obtained in S5 to generate the final embedding; S7, according to the final embedding output by S6, use Wasserstein distance to calculate the distance between the distributions of two items in the user interaction sequence as the ranking score to generate the recommendation list; Use Bayesian personalized ranking loss as the basic loss function, and introduce the regularization term considering the distance between positive and negative sampling items and the global information coding loss to optimize the basic loss function for the recommendation task.

2. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 1, characterized in that, The specific steps of S1 are as follows: S1.1, define a user's interaction sequence update the weight of the edge between items and item in the sequence to where denotes the th item in the user's interaction sequence, denotes the th item in the user's interaction sequence, and score denotes the importance of the target node to its k-hop neighbors ; S1.2, repeat S1.1 for all users, build global project graph Then normalize the edge weights between two nodes and ​ ； wherein, is an adjacency matrix, is a normalized adjacency matrix, is the degree of a node in S1.3, to SVD processing is performed to obtain an enhanced image The expression of SVD is: ； wherein, and is a standard orthogonal matrix, is a diagonal matrix; denotes the normalized adjacency matrix; S1.4, capture the global information in the form of a graph, use the graph encoding method of LightGCN to obtain the representation of the graph, and the expression of the graph embedding is: ； wherein, is the node embedding of a layer, denotes the number of layers of the graph convolution, denotes the adjacency matrix, denotes the normalized adjacency matrix.

3. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 1, characterized in that, The specific steps of S2 are as follows: For all items, the items are described by a multivariate elliptical Gaussian distribution, first define a mean embedding table and a covariance embedding table where d denotes the number of latent dimensions, and V denotes the number of items; then the sequence embedding of the mean and covariance of the items in the user interaction sequence is as follows: ； wherein, a sequence embedding representing a mean of items, a sequence embedding representing a covariance of items, a mean embedding representing the n-th item in a sequence of user interactions, a covariance embedding representing the n-th item in a sequence of user interactions, a n-th item of a table of mean embeddings, a n-th item of a table of covariance embeddings.

4. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 3, characterized in that, The specific steps of S3 are as follows: S3.1, Obtain a temporal query embedding as follows : ， wherein, is a normalization factor ensuring the sum of weights is 1 ; is the exponential decay weight for the th element, is a hyperparameter for controlling the attention query range; denotes the embedding at time t. S3.2, use Wasserstein distance to measure the distance between two random embeddings; Items in a given user's interaction sequence and The corresponding random embeddings are and where , , , and are the mean embedding and covariance embedding of the i-th item in the user interaction sequence, respectively, and are learnable parameters, ELU denotes the activation function, and diag denotes extracting the diagonal matrix; the attention weight is defined by measuring the negative 2-Wasserstein distance , and the position information is added to the calculation of the attention weight in the form of bias, and the attention weight is obtained, and the calculation formula is as follows: ； wherein, is a 2nd order Wasserstein distance, is an attention weight, is a relative distance matrix, denotes a position index, is a tuning parameter, set according to the number of attention heads; S3.3, each item is represented by a random embedding containing mean and covariance, and the linear combination property of Gaussian distribution is used to aggregate information, and the formula is as follows: ； wherein, is the normalized attention weight value, and respectively represent the value vector of the mean and the value vector of the covariance, both of which are obtained by linearly transforming the initial sequence embedding; represents the new sequence mean embedding after aggregating information, represents the new sequence covariance embedding after aggregating information, wherein j takes values from 1 to n.

5. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 4, characterized in that, The specific steps of S4 are as follows: S4.1, introduce the feedforward network layer to the new sequence embedding output in S3, capture more complex relationships, and the feedforward network layer first passes through a multilayer perceptron, and then passes through a linear layer, and the formula is as follows: ； wherein, and are the mean output and the covariance output of the feedforward network layer, respectively, are all learnable parameters. S4.2, use residual connection, layer normalization and random loss operation to update the output of the feedforward network layer to: ； wherein, denotes the updated sequence mean embedding, denotes the updated sequence covariance embedding.

6. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 1, characterized in that, In S5, the new sequence embedding output by the feedforward network layer in S4 is embedded according to the length of the time window The method for defining the short-term sequence embedding and the long-term sequence embedding is as follows: the output of the time feedforward network layer, for a short-term sequence embedding; the output of the time feedforward network layer, for a long-term sequence embedding.

7. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 1, characterized in that, The specific steps of S6 are as follows: S6.1, the formula for obtaining the new short-term sequence embedding is as follows: ； wherein, In denotes the short-term sequence embedding obtained from step five, denotes the graph embedding obtained from step one, and these two parts are combined layer to obtain the updated ; S6.2, the method of integrated query embedding, the new short-term sequence embedding obtained in step 6.1 is combined with the long-term sequence embedding to generate a final embedding; the formula is as follows: ； wherein is a new short-term sequence embedding; is a long-term sequence embedding; is a learnable hyperparameter.

8. The sequence recommendation method based on global information and hybrid time query attention mechanism according to claim 1, characterized in that, The specific steps of S7 are as follows: (1) Compute the Wasserstein distance between the distributions of two items to predict the ranking score, for item , the predicted score of the next item is computed as follows: ； (2) Bayesian personalized ranking is used as a basic loss function, and a regularization term considering the distance of positive and negative sampling items and a global information coding loss are introduced to optimize the basic loss function; Regularization term The formula is as follows: ； wherein, , is the next item, i.e. the positive item, accessed by the user, is a negative item randomly sampled from the items that have never been interacted with from the user interaction sequence; denotes the t-th item in the user interaction sequence; denotes the distance between the t-th item in the user interaction sequence and the positive item, denotes the distance between the positive item and the negative item; the global information encoding loss formula is as follows: ； wherein, is the original global item graph is the embedding obtained by graph encoding, is the augmented graph is the embedding obtained by graph encoding, denotes the number of samples; The final loss function is: ； wherein, denotes a sequence of interactions of a user, denotes a logistic function, is the amount of all learnable parameters, is a hyperparameter, and are two hyperparameters for controlling the two constraint terms.

Images