Research on graph-coupled time interval network for sequential recommendation

Abstract

In recent years, the Transformer framework and graph neural networks have attracted great attention from academia and industry. Some researchers have used the Transformer framework and the original user-item interaction graph to improve the user representation learning process. In order to improve the accuracy of serialized recommendations, the present invention further regards category information as a new type of node, and in this way explicitly encodes collaborative signals in the user-item-category tripartite graph; at the same time, in order to explicitly use time interval information to assist in predicting the next item, the present invention discloses a time interval-sensitive self-attention mechanism to capture the time interval information between recent behaviors and target items. Combined with actual scenarios, in order to eliminate the semantic gap between collaborative information and sequence information, the present invention discloses a personalized gating strategy to distinguish the contribution of each module in different scenarios.

Description

[0001] Technology Space

[0002] This invention relates to the fields of machine learning and recommendation systems, and in particular to the research on graph neural networks and Transformer frameworks in the field of sequential recommendation. Background Art

[0003] With the development of the internet and the popularization of mobile devices, the internet and electronic products have brought convenience to people, but have also led to the emergence of information overload. Personalized recommendations, as an important tool to alleviate this problem, have been deployed in applications such as streaming short videos and e-commerce.

[0004] Collaborative filtering (CF), the most popular technique among traditional recommendation methods, models a bipartite user-item graph and uses user ratings or other behavioral patterns to provide personalized recommendations. While simple and effective, CF-based recommendation methods ignore the rich auxiliary information inherent in the model and model user interests in a static manner. In real life, because user interests are inherently dynamic, inferring which item a user will visit next is more practical than understanding their general preferences. This problem is also known as sequential recommendation. For example, if user A first purchases an iPhone and then AirPods headphones, it is likely that they will then browse phone cases or screen protectors. Therefore, many current studies leverage the dynamic nature of user-item interactions to model user interests and improve recommendation accuracy.

[0005] In recent years, with the widespread success of graph neural networks and the Transformer framework in academia and industry, researchers have focused on deploying them in recommendation system tasks. Graph neural networks leverage user-item bipartite graphs, stacking multiple representation propagation layers to encode collaborative signals from the interaction graph into the representation learning process. For the Transformer framework, the self-attention mechanism eliminates complex recursive or convolutional operations, achieving significant performance and efficiency improvements. However, the inventors found that previous work still has limitations in the following three aspects:

[0006] (1) Relying solely on individual behavioral data to model user long-term interests without explicitly utilizing collaborative signals and other auxiliary information. However, item popularity follows a long-tail distribution, and GNN-based methods rely heavily on the performance of user-item interaction graphs. However, inactive users and unpopular items have only limited neighbors, which makes it difficult for traditional GNN-based models to learn satisfactory user long-term preferences.

[0007] (2) They ignore the influence of the time interval between multiple recent behaviors and target items. They usually regard the user's behavior records as an ordered sequence and cannot perceive the time interval information between recent behaviors and target items, which may lead to the inability to accurately model the user's short-term interests.

[0008] (3) Lack of adaptability. After obtaining the user's long-term and short-term interests, existing works usually use addition or splicing operations to make predictions. Although these two methods are simple and intuitive, they cannot distinguish the contribution of each part in a specific context. Summary of the Invention

[0009] To address these issues, the present invention discloses a graph-coupled time interval network for sequential recommendations, comprising four modules: a data input module, a category-aware graph propagation module, a time interval-sensitive self-attention mechanism, and a model prediction module. First, based on initial representations of user, item, and category IDs and mapping them into a D-dimensional vector space, a representation lookup table is established for all users, items, and categories to obtain the corresponding input matrix. Second, to improve the initial representations of all users and items, a category-aware graph propagation module is designed. This module treats category information as a new type of node and integrates user-user and user-item proximity information of similar categories into the graph structure, thus expanding the original user-item interaction graph structure. Based on this, we explicitly mine high-order neighbor information in the user-item-category trigraph to capture behavioral similarities between users. We then extract the most recent L interactions as input for short-term interest modeling. We first inject user representations into the behavior subsequence representation matrix and, using a designed time interval calculation method, obtain personalized time interval information to overcome the drawback of previous work that implicitly assumes that any two behaviors have the same time interval. Furthermore, we utilize a time interval-sensitive self-attention mechanism to filter out items that are more important for next-moment recommendations, thereby deriving a user's short-term interest representation. Finally, we employ a personalized gating strategy to fuse the two representations obtained through the category-aware graph propagation module and the time interval-sensitive self-attention mechanism. This strategy adaptively assigns weights to the two components based on different recommendation scenarios, ultimately yielding each user's final interest representation.

[0010] Compared with the prior art, the effective effects of the present invention are:

[0011] (1) This paper proposes a graph-coupled time interval network for serialized recommendations, aiming to focus on item category and time interval information to improve the quality of personalized recommendations. It seamlessly combines the advantages of graph neural networks and Transformer frameworks to jointly explore the importance of these two types of information for predicting items at the next moment.

[0012] (2) This paper can capture users' long-term and short-term interests in a fine-grained manner. It designs a category-aware graph propagation module and a time interval-sensitive self-attention mechanism. The former learns to balance the representation of popular nodes and tail nodes through the high-order connectivity in the user-item-category tripartite graph; the latter explicitly models the impact of time interval information on the next moment of recommended items.

[0013] (3) This paper designs a personalized gating strategy to dynamically distinguish the contribution of each module in different scenarios. This strategy enables the public model to not only improve the accuracy of recommendations but also improve training efficiency, achieving an optimal balance between accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or description of the prior art.

[0015] Figure 1 is a schematic diagram of a recommended scenario according to an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of a GCTN framework according to an embodiment of the present invention;

[0017] Figure 3 This is a comparison chart of the evaluation index results of the GCTN embodiment of the present invention under different characterization dimension settings;

[0018] Figure 4 This is a comparison chart of normalized cumulative loss gains of various recommendation methods under different recommendation list length settings according to an embodiment of the present invention;

[0019] Figure 5 This is a comparison chart of the accuracy of various recommendation methods under different recommendation list lengths according to an embodiment of the present invention;

[0020] Figure 6 is a comparison chart of normalized cumulative loss gains for different user groups according to an embodiment of the present invention;

[0021] Figure 7 It is a comparison chart of the training time of the present invention and the benchmark method.

[0022] Specific implementation

[0023] With the rapid development of the Internet, online shopping has gradually become a mainstream shopping method. In the item search interface of the Taobao mobile app, the system gives "Guess what you want to search" keywords and "Taobao" hot search recommendations based on the user's previous historical searches.

[0024] like Figure 1The figure below illustrates a user recommendation example in an e-commerce scenario, where user interests and preferences are represented by information at different time intervals. For example, user Mia's purchase history is {(handbag, April 3), (skirt, April 26), (jeans, May 1), (jewelry, May 30)}; user Linda's recent purchase history is {(handbag, May 22), (shorts, May 27), (slippers, May 28), (sunglasses, May 31)}. Although the item "handbag" occupies the same ordinal position for both users, it clearly contributes more information to Linda's short-term interests. When modeling users' long-term and short-term interests, whether or not auxiliary information is considered directly impacts the accuracy of recommendation results. Similarly, how time interval information is utilized will significantly impact the prediction of items at the next moment.

[0025] 1. Problem Definition:

[0026] For serialized recommendations, use and Represent the sets of users and items respectively, C represents the set of item categories, where each item belongs to a category, Indicates whether user u has clicked / purchased / visited the corresponding item, r ui =1 means that user u and item i have interactive behavior, otherwise r ui =0. and Represents the number of users and the number of items respectively. According to the user-item interaction matrix R and the item category set C, the user-item-category interaction graph can be constructed The symbol ε represents the edge set between nodes. The input data of the present invention is mapped to a D-dimensional vector space according to users, items, and categories, which can be formally expressed as: in and use Captures the serialization schema.

[0027] Given a user-item-category tripartite graph and users Behavior sequence Predicted Rating Indicates user u’s preference for item i at time t. Based on this prediction, the set of items that user u has not visited is Provides a ranked list of items that may be of interest to them.

[0028] 2. Overall architecture of the model:

[0029] Figure 2The overall architecture of the GCTN is presented. The network consists of four modules: a data input module, a category-aware graph propagation module, a time interval-sensitive self-attention mechanism, and a model prediction module. The following sections describe the specific implementation of the network architecture involved in this invention:

[0030] 2.1 Data input module:

[0031] The initial representations obtained based on users, items, and categories are used as the input data of the network. In order to capture the serialization mode, vector Each item is additionally encoded, and the specific representation is:

[0032]

[0033] 2.2 Category-aware graph propagation module:

[0034] Construct message propagation rules on the user-item-category tripartite graph. For a user-item-category triple (u, c, i), define the message propagation rule from user u to item i at the kth layer as:

[0035]

[0036] Injecting user u’s topic interest in item i into the representation learning process is formally defined as:

[0037]

[0038] in is a symmetric normalization term to avoid excessively large values, and ⊙ represents the element-wise product of two vectors. Similarly, the message propagation rule from item i to user u can be obtained:

[0039]

[0040] After performing k neighborhood aggregation operations, we obtain the node representations of each graph propagation layer. To enrich the model capabilities, we use the weighted sum aggregator to integrate these representations to construct the final node representation:

[0041]

[0042] 2.3 Time interval sensitive self-attention mechanism:

[0043] In order to mine sequence patterns from a user's recent multiple behaviors and capture the time interval between each behavior and the user's current state, it is very necessary to explore the impact of time interval information on predicting the user's interest at the next moment. In view of this, the present invention designs a time interval sensitive self-attention mechanism to model the user's short-term interests. The mechanism consists of a time interval perception representation, a time interval sensitive self-attention layer, and a feedforward layer. First, the length of the user behavior sequence is adjusted. If the number of behaviors in the sequence is less than L, the rightmost end of the sequence is filled with a full zero vector until |S u |=L; conversely, the most recent L interactions are intercepted to form a behavior sequence. From this sequence, the representation matrix can be obtained Further user representation vector Embedding, obtaining a personalized representation matrix

[0044]

[0045] in Represents the addition operation of the corresponding elements of the vector. Assume that the current timestamp of user u is T t u , access items The time is The time interval between the two can be defined as:

[0046]

[0047] Based on the user's recent behavior sequence, the corresponding time interval matrix can be obtained Inject it into The representation matrix of time interval perception is obtained:

[0048]

[0049] In order to obtain high-quality short-term interest representation, a time interval sensitive self-attention mechanism is introduced. Specifically, Converted into three matrices, namely the query matrix R, the key matrix K and the value matrix V, formally defined as:

[0050]

[0051] The corresponding dot product attention is defined as:

[0052]

[0053] Although the time interval-sensitive self-attention layer can adaptively aggregate the representations of recently interacted items, it is difficult to capture complex item translation patterns because it only captures linear relationships. Therefore, a forward feedback layer is further used to explore the interactions between different factors:

[0054] F=FFN(S)=ReLU(SW ′+ b′)W′ ′+ b″ (11)

[0055] in is a trainable network parameter, and F represents the aggregate representation of multiple recently interacted items. The present invention employs Dropout regularization, layer normalization, and residual connections to avoid overfitting and vanishing gradient problems. Iterating the above operations b times yields:

[0056] S (b) =SA(F (b-1) ),F (b) =FFN(S (b) ) (12)

[0057] Among them S (1) =S,F (1) =F. After executing b times, the vector To express the user's short-term interests.

[0058] 2.4 Model Prediction:

[0059] How to integrate a user's long-term and short-term interests directly impacts the performance of sequential recommendations. Additive and concatenated strategies are two common options, but they fail to distinguish the contributions of different components to different users. For example, for a stubborn man, his long-term preferences are more important for predicting future behavior, while for a fickle woman, her short-term interests are more important. Even for the same user, the contribution of each component varies at different times. To address this, a personalized gating strategy is designed to adaptively assign weights to these two components to derive user u's preferences at time t:

[0060]

[0061] in is a learnable weight matrix. Finally, the probability of user u accessing item i at time t is:

[0062]

[0063] 3. Model optimization process:

[0064] Given user u and the set of items visited before time t The present invention uses the cross entropy loss function to learn network parameters, which can be formally calculated by maximizing the likelihood function value:

[0065]

[0066] By taking the negative logarithm of the above likelihood function, the objective function of GCTN can be written as:

[0067]

[0068] Θ1={E,M,P},Θ2={H,w x ,b x ,W Q ,W K ,W V ,W g ,W′,W″,b′,b″} is the network

[0069] The network parameters are set, and a negative sample j is randomly selected for each target item i following the negative sampling strategy.

[0070] 3.1 Complexity Analysis:

[0071] (1) Space complexity: The trainable network parameters in the GCTN proposed in this paper are composed of two parts: Θ1 and Θ2. The first part comes from the initial representation of users and items: The second part is independent of the number of users and items: (|C|+6)D+5D 2 The network size of GCTN is comparable to that of the most advanced HGN. And it is much smaller than the model size of classic FPMC

[0072] (2) Time complexity: The time complexity of the present invention mainly comes from the category-aware graph propagation module and the time interval-sensitive self-attention mechanism. The time overhead of the former comes from the final representation and The time complexity of the generation of is O(DK|ε|), where K is the number of graph convolution layers and |ε| is the total number of all user behaviors. The latter involves self-attention operations and forward feedback layers, and its time complexity is: O(L 2 D+LD 2 ). Therefore, the time complexity of each iteration of GCTN is: O(DK|ε|+L 2 D+LD 2 ).

[0073] Figure 3 This is a comparison chart of the results of various recommendation methods under different representation dimension settings when the length of the recommendation list is fixed to 10 in the example of the present invention. Figure 4 This is a comparison chart of the accuracy of various recommendation methods under different recommendation list lengths according to an embodiment of the present invention. Figure 5 This is a comparison chart of the normalized cumulative loss-to-gain of various recommendation methods for different recommendation list lengths, with higher values ​​indicating better performance. By comparing the recommendation performance evaluation index values ​​of each method, GCTN achieved the best performance under both benchmark evaluation metrics. Figure 6This is a comparison chart of recommendation performance on different user groups according to an embodiment of the present invention. Figure 7 This is a comparison chart of the training time of various recommendation methods provided by the embodiments of the present invention. GCTN has higher training efficiency.

[0074] Those skilled in the art will readily understand the embodiments of the present invention. The above description is merely an example of a basic implementation of the present invention and is not intended to limit the present invention.

Claims

1. A serialization recommendation method based on graph-coupled time interval network, characterized in that: (1) Focusing on the modeling process of users' long-term and short-term interests in sequential recommendation, we use item category information and time interval information to jointly explore the importance of predicting the next item; (2) A personalized gating strategy was designed to distinguish the contribution of each module in different recommendation scenarios and improve the model prediction accuracy. (3) This method consists of the following four modules: ① Data input module: Map the IDs of users, items, and categories into a D-dimensional vector space to obtain three representation matrices for users, items, and categories: ②Category-aware graph propagation module: Step 1: For each user-item-category triple (u, c, i), the message propagation rules from user u to item i and the message propagation rules from item i to user u at the kth level are: Step 2: Use average weighting to generate the final representation of each user and item node: ③Time interval sensitive self-attention mechanism: Step 1: Select the user's most recent L interactions to form a behavior sequence User representation vector Inject into matrix O u In the above example, we get the personalized representation matrix Step 2: Assume the current timestamp is T t u , access items The time is The time interval characteristics between the two are: Step 3: Inject time interval features into the matrix The representation matrix of time interval perception is obtained from Step 4: Converted into three matrices, namely the query matrix R, the key matrix K and the value matrix V: Step 5: Input R, K, V into the attention operation: Step 6: Use a feedforward layer to explore the interactions between different factors: F=FFN(S)=ReLU(SW′+b′)W″+b″ Step 7: Perform b attention layers and forward feedback layers: S (b) =SA(F (b-1) ),F (b) =FFN(S (b) ) ④Model prediction: Step 1: The user's short-term interests are expressed as: Step 2: User u at time t is represented as: Step 3: Calculate the probability that user u visits item i at time t: .

2. The method for sequential recommendation based on graph-coupled time interval network according to claim 1, characterized in that: Step 1: Use binary cross entropy loss function to optimize model parameters: Step 2: Minimize the negative logarithm of the binary cross entropy: 。

Images