A transformer-based cross-sequence multi-behavior sequence recommendation method

Abstract

The application discloses a cross-sequence multi-behavior sequence recommendation method based on a Transformer. The method is as follows: firstly, a user historical interaction sequence is segmented, and then cross-sequence heterogeneous relationship propagation is performed; then, short-term interaction mode learning is performed; next, global representation aggregation is performed; finally, a recommendation content list is generated by combining user recent preferences and item embedding, and meanwhile, content priority is considered; and model parameters are adjusted by using user historical interaction for label data enhancement. The application proposes a user sequence segmentation method, guarantees that a subsequence maintains relatively concentrated and stable preferences of a user, and enables the model to more accurately capture dynamic changing preferences of the user, proposes a similar subsequence determination mechanism and a method for performing cross-sequence information propagation between similar subsequences, and preserves time coding information in the information propagation process, thereby improving the diversity and accuracy of recommended contents.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of recommendation system, and particularly relates to a cross-sequence multi-behavior sequence recommendation method based on a Transformer. BACKGROUND

[0002] In order to alleviate the problem of information overload caused by the rapid development of information technology, the concept of a recommendation system aiming to provide personalized content recommendation is proposed. Since the preferences of users usually evolve over time, a sequence recommendation system modeling the dynamic preferences of users according to their historical interaction behaviors emerges as the times require and achieves remarkable results, gradually becoming a hot research issue.

[0003] In recent years, with the great success of deep learning technology in many fields, people have begun to focus on using various neural network architectures to model user-item interaction records and mine the interest evolution process of users to solve the sequential recommendation problem. Existing sequence recommendation researches are mostly focused on modeling single type of interaction behavior records. In actual applications, the interaction types between users and items are usually diverse, and different behaviors reflect different degrees of user preferences, for example, the possibility of a user purchasing an item that has been added to the shopping cart is higher than that of a product that has only been browsed. Related researches have attempted to distinguish the potential semantics of complex interaction types. Early methods have predefined cascading relationships, and now the mainstream methods have learned user preferences from multi-behavior data using recurrent neural models and graph neural network models.

[0004] Existing multi-behavior recommendation works have achieved certain results in mining the preference degree reflected by complex behaviors, but they are all focused on mining the interaction sequences of users themselves, ignoring the complementary information provided by the interaction sequences of other users with similar interests, and the information utilization of interaction time is insufficient, which makes it difficult to capture the dynamic preferences of users evolving over time. Therefore, how to cut the historical interaction sequence of users so that each sub-sequence presents relatively stable and concentrated preferences of users is a problem to be solved, and secondly, how to mine user interaction information in a global view and perform information propagation is of great significance to improve the performance of the recommendation system. SUMMARY

[0005] In order to solve the above problems, the present application provides a cross-sequence multi-behavior sequence recommendation method CMB (Cross sequence Multi-Behavior Sequential Recommendation based on Transformer) based on a Transformer, which can accurately capture long-term and short-term preferences of users and realize accurate service content recommendation.

[0006] In order to achieve the above object, the technical scheme of the present application is as follows: a cross-sequence multi-behavior sequence recommendation method based on a Transformer, characterized by comprising the following steps:

[0007] Step 1, segmenting a user historical interaction sequence;

[0008] Step 2, cross-sequence heterogeneous relationship propagation;

[0009] Step 3, short-term interaction mode learning;

[0010] Step 4, global representation aggregation;

[0011] Step 5, generating a recommendation content list by combining user recent preferences and item embeddings, while considering content priorities, and adjusting model parameters by using user historical interactions for label data enhancement.

[0012] The present application has the following advantages: compared with the existing multi-behavior recommendation method, the present application solves the problems of insufficient utilization of time context information, inaccurate capture of user dynamic preferences, and neglect of user interaction information mining in a global view. The present application has the following advantages: (1) a user sequence segmentation method is proposed, which ensures that the subsequence maintains the user's relatively concentrated and stable preferences, so that the model can more accurately capture the user's dynamic changing preferences. (2) a similar subsequence determination mechanism and a cross-sequence information propagation method between similar subsequences are proposed, and the time coding information is preserved during the information propagation process, which improves the diversity and accuracy of the recommended content. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The present application is a general framework diagram of a cross-sequence multi-behavior sequence recommendation method based on a Transformer.

[0014] Figure 2 The present application is a user interaction sequence segmentation diagram.

[0015] Figure 3 The present application is a similar subsequence diagram.

[0016] In the drawings, different colors are used to distinguish between behavior types and different channel parameter matrices in the multi-channel projection layer. DETAILED DESCRIPTION

[0017] Further, the step 1 of segmenting a user historical interaction sequence is as follows:

[0018] The present application takes an e-commerce platform as an example, and refers to behaviors such as browsing and collecting, which reflect user preferences, as auxiliary behaviors, and refers to behaviors such as purchasing, which the platform hopes users to do and which generate revenue for the platform, as target behaviors. Based on this scenario, the user set is written as U={u1,…,u i,…,u I}, the project set is written as V={v1,…,v j ,…v J The various interactive behavior types that appear in the scene are defined as set B (indexed byb), which contains all possible behavior types (such as browsing, marking as a like, adding to cart, and purchasing).

[0019] Where I and J∈Z represent the number of users and the number of projects, respectively.

[0020] In the context of this invention, each user's historical interactions are recorded as a multi-behavior interaction record. Specifically, for a user u i ∈U, whose historical interaction information is defined as a triple (v i,x b i,x , t i,x ), representing user u i At time x, b i,x The xth item of type behavior interaction constitutes user u i Multi-behavior interaction records, denoted as S i ={(v i,1 b i,1 , t i,1 ), ..., (v i,x b i,x , t i,x ), ..., (v i,X b i,X , t i,X )}.

[0021] Where X represents the sequence length |S i |

[0022] To model users' preferences that evolve over a short period of time, we consider segmenting users' interaction records and encoding them in conjunction with contextual information. Then, to enhance information flow and improve the diversity of the recommendation system, we designed a cross-sequence information transfer component, which allows users to obtain information about items they have interacted with from similar interaction records of other users.

[0023] First, the user's interaction sequence is divided into multiple subsequences using the Seg(·) operation. For a given user u i and its interaction sequence S i The Seg(·) operation can be expressed as formula (1):

[0024]

[0025] Where P (indexed by p) represents the number of user subsequences segmented, b target Indicates the target behavior. and u i the length of the first and last subsequence, denotes the maximum length of subsequence.

[0026] Specifically, the process integrates all users' interaction sequences, and divides the users' interaction sequences into several subsequences with the items that the users make target behavior (e.g. purchase) as the cut-off point. The reason for doing so is that the items that the users interact with before purchasing an item have different degrees of similarity or relevance, and the user interaction subsequence reflects relatively concentrated and stable preferences before the user purchases a product. With such sequence segmentation, the user's subsequence helps to accurately capture the user's short-term preferences. If the user has not made a target behavior, in order to analyze the user's short-term preferences, the maximum subsequence length Segmenting users' historical interaction sequences.

[0027] Under the global view, if the last item of all users' segmented interaction subsequences is the same item that is interacted with the target behavior, then the two subsequences are considered similar. For example, given user u i the kth subsequence of u User u i′ has a subsequence If there are then the sequence is similar to the sequence denoted as Each subsequence can have multiple similar subsequences.

[0028] Further, the cross-sequence heterogeneous relationship propagation described in step 2 is as follows:

[0029] This method designs an embedding network that jointly incorporates behavior type information and time signals as context information into the feature representation of the item. First, following the mainstream recommendation paradigm, the initial embedding e j ∈R d is generated by embedding the item set V into the same space, and the item embedding matrix M ∈ R |I|×d is obtained, which is related to the item id.

[0030] where d denotes the dimension of the latent state. Then, e b ∈R d is associated with the user behavior type, representing the bth type of behavior embedding.

[0031] In addition, in order to give the model time perception ability, this method introduces a time encoding strategy in the context information encoding layer, i.e. using formula (2) to map the scalar timestamp to the vector space:

[0032]

[0033] Where τ(·) represents the function that maps timestamps to 2d-dimensional vectors, W t ∈R 2d×d The time transformation matrix is ​​represented by 2l and 2l+1, respectively, where odd and even indices of the 2d-dimensional vector are denoted by 2l and 2l+1. Furthermore, to mitigate the impact of large embedding values, [the following is used]. As a scaling factor, W t The role of the matrix is ​​to introduce an adjustable linear projection mechanism, giving the model the ability to flexibly learn temporal context information.

[0034] Following this, the behavior type representation and time context information are jointly injected into the project representation using the following operations:

[0035]

[0036] in This indicates that the item embedding incorporates contextual information and serves as input to the cross-sequence information passing component. This indicates an element-wise addition operation on a vector.

[0037] To capture the dynamic similarity between different items in the sequence view and the global view, this method designs a cross-sequence information propagation encoder based on the Transformer architecture and an attention mechanism in a multi-head representation space.

[0038] Specifically, the embedding of the x-th interaction term in the sequence is calculated using the multi-head dot product attention method as shown in Equation (4):

[0039]

[0040] in This represents the user u calculated after cross-sequence information aggregation. i A refined representation of the x-th interaction term in the k-th subsequence, where Y represents the item. Related item set (i.e., sequence) Other items and sequences in Items in similar sequences), E y Represents the embedding representation of related terms, α x,y Projects learned and v y The correlation coefficient between them, σ(·) represents the softmax function, h∈H represents the h-th head of the attention layer, Q h K h , Let represent the Query, Key, and Value transformation matrices of the h-th head (three important parameter matrices in the attention mechanism), and divide the query and value vector scores by . with a stable gradient.

[0041] The introduction of the multi-head attention mechanism enables the encoder to perform information aggregation of different semantic dimensions under multiple representation subspaces. The cross-sequence heterogeneous relationship propagation module, by integrating the context information encoding layer and the cross-sequence information propagation layer, not only enables the model to be sensitive to the user interaction context information, but also captures the dynamic characteristics of multiple behaviors, and preserves the transition relationship between items under the sequence view and the global view.

[0042] Further, the short-term interaction pattern learning in step 3 is specifically as follows:

[0043] The behavior-aware encoder is designed to aggregate specific types of behavior patterns in the short term, and to give the model the ability to distinguish the heterogeneity of intra-sequence behaviors. For a given user u i and the sequence u i The b-th type of behavior pattern in the corresponding short term can be formally expressed as formula (5):

[0044]

[0045] wherein represents the aggregated calculation of u i The feature representation of the b-th behavior in the epoch, ρ(·) is a judgment function defined by the method, which is 1 if the two input items are equal, and 0 otherwise, is the item refined embedding representation output by the cross-sequence heterogeneous relationship propagation module, W b is the behavior-aware transformation matrix, which is used to enhance the semantics of specific types of behaviors, and its optimization path is based on the multi-channel parameter learning strategy shown in formula (6):

[0046]

[0047] wherein M represents the number of channels (indexed by m), P∈R M×d and μ∈R M represent the transformation matrix and the bias parameter in the fully connected layer, respectively, represents the transformation matrix corresponding to the m-th channel. The purpose of introducing the multi-channel parameter learning strategy is to preserve the heterogeneity of interaction behaviors in the message passing process, and to use the channel-based aggregation layer to encode the behavior-aware semantics, so that the CMB model can capture the potential semantics of different types of user behaviors.

[0048] After obtaining the embedded representation of a specific behavior type, the short-term behavior type encoding needs to be fed into the aggregation layer to capture the potential interdependencies between different behavior types, thus obtaining the user's short-term feature representation. Unlike the static attention relationship aggregation scheme used by other recommendation models, this method uses an adaptive attention network to calculate the contribution index of each behavior type to the short-term representation. The aggregation layer operation can be formalized as follows:

[0049]

[0050] in User u i The feature representation of the k-th subsequence, i.e., u i In the short-term preference encoding within the corresponding period, β i,b This means that the sum of all behavior embeddings is used as the u learned by the query. i The attention weight corresponding to behavior b, W A and μ A These are the transformation matrix and bias of the aggregation layer, respectively.

[0051] After the aggregation layer, the user's short-term representation The system explicitly preserves temporal and behavioral contextual information.

[0052] Accordingly, for the message passing process from the user to the project side, in order to refine the behavior perception embedding of the project, the embedding code of the project is also subjected to the above propagation and aggregation process, as shown in formula (8):

[0053]

[0054] in and E j They represent project v respectively j Specific behavior type embedding and fused behavior-aware embedding. This scheme can distinguish the impact of user behavior on projects and incorporate time signals into the message passing process, because the computation during propagation involves user subsequence feature encoding.

[0055] Furthermore, the global representation aggregation described in step 4 is as follows:

[0056] User preferences are inherently dynamic and evolve over time. Therefore, the CMB model further proposes a global graph relation encoder to inject the embedded representations of users at various time periods into the global feature representation, capturing the dynamic patterns of users' multi-behavior behavior from both long-term and short-term perspectives. This method defines a global user graph to contain all user nodes u. i ∈U and their interactive subsequence nodes i.e., G = {U, S, ε}, where S and ε represent the set of all user sub-sequence and the connection between each user u i with his interaction sub-sequence node edges (1 < k < K), the global representation aggregation module performs graph structure information aggregation on the global user graph G to generate the global embedding representation of users, and incorporates temporal context information in the information aggregation process, so that short-term representations of different periods interact in a distinguishable way, the calculation process is as shown in formula (9):

[0057]

[0058] where represents the global feature representation of user u i , σ(·) represents the softmax activation function, and η k represents the learned attention weight of user u i to its kth sub-sequence, is the time encoding returned by the context information encoding layer, corresponding to the interaction time of the last item in the sequence .

[0059] By doing so, the CMB retains the user preference embedding of each period and incorporates a unified global user feature embedding representation.

[0060] After the above process, the CMB framework performs information aggregation and encodes the user's short-term interaction behavior patterns and long-term dynamic user embedding representation. In the embedding propagation process, the CMB takes the user-item interaction graph and the global user graph G as the calculation graph, during which the local relationship features from the neighborhood will be aggregated to obtain the context representation. Such a message passing paradigm can be generalized to formula (10) by merging high-order connectivity (l represents the lth layer graph neural network in the graph message propagation process):

[0061]

[0062] where and are the item and user embeddings initialized by the encoding layer, respectively. By extending the L-layer graph relationship encoder, the CMB is given the ability to capture high-order neighbor collaboration signals. Finally, the final user item and sub-sequence feature representations are obtained by corresponding addition operation of L-layer embedding elements, the process of which can be formalized as (11):

[0063]

[0064] Further, the joint user recent preference and item embedding generates a recommendation content list, while considering content priority, by adjusting model parameters through label data enhancement using user historical interactions, as follows:

[0065] In the prediction layer of the model, in order to reflect the dynamic characteristics of user preferences, the method jointly uses the short-term feature representation of the last stage of the user and the item feature representation E j To predict the possible interaction items of the user in the future, the user u i and the item v j The possibility of interaction can be quantified as Where is a parameter vector, represents the element-wise multiplication operation, in the training stage, the parameter adjustment process is optimized by using label data enhancement through user historical interactions, so that u i and v j The prediction score in the corresponding period of can be calculated as: In this case, the objective function of model training and marginal pair loss can be formalized as follows:

[0066]

[0067] Where I and K i represent the number of users and the number of interaction sub-sequences owned by the user u i , C represents the sampling number (indexed by c), p c and n c represent the corresponding positive samples and negative samples, and the method further introduces a weight decay regularization term to alleviate the overfitting phenomenon.

[0068] Experimental process:

[0069] a) Dataset

[0070] In order to study the performance of the CMB model in real scenarios, the present application collects two public datasets from two e-commerce platforms in the real world for experimental evaluation (see Table 1), both datasets contain multiple behavior types of users.

[0071] Table 1

[0072] Dataset details

[0073] Dataset name Number of users Number of projects Number of interactions Number of behavior types Taobao 147894 99037 7658926 4 IJCAI 423423 874328 36203512 4

[0074] Taobao: Taobao is one of the largest e-commerce platforms in Asia, providing services such as guaranteed transactions. This dataset contains user-item interaction records collected from the Taobao retail system over a period of time, covering four types of interactions: page views, adding to the shopping cart, marking as favorite, and purchasing.

[0075] IJCAI-Contest: This dataset records user interaction records on an online e-commerce website, also containing four types of interaction behaviors: clicks, adding to the shopping cart, marking as favorite, and purchasing. The dataset is from the IJCAI contest.

[0076] The experiment first formats each record in the dataset into user id, item id, behavior type, and timestamp, and generates interaction sequences S according to the chronological order of user interaction timestamps. i Considering the profitability of e-commerce platforms, "purchase" is set as the target behavior, and other types of user behavior are considered as auxiliary behaviors that provide supplementary information for the target behavior.

[0077] b) Evaluation indicators

[0078] Following the mainstream evaluation method of recommendation systems, two widely used indicators are used: hit rate (HR@N) and normalized discounted cumulative gain (NDCG@N) to evaluate the performance of the CMB model and the baseline model, thus measuring the accuracy of the top-N recommendation results provided by these methods for different users. The performance of the model is proportional to the NDCG and HR scores. HR@N represents the percentage of items in the top-N list recommended to the user among the test set items, reflecting the accuracy of the recommendation results. NDCG@N calculates the relevance of the recommendation results at each position in the recommendation list, which not only considers the relevance of the recommended items, but also considers the position of the recommended items. This is because the ranking position of items has a great influence on user experience in recommendation systems, and the NDCG score can reflect the quality of the ranking of recommendation results. For the training and test data samples generated based on the time information of user interaction data, the leave-one-out method is used for evaluation, that is, for each user in the test set, the last interaction item in the dataset is used as a test instance for recommendation.

[0079] c) Comparison methods

[0080] [1] Neural network combined with collaborative filtering model:

[0081] NCF: Introducing neural networks to learn user and item interaction information, unlike traditional matrix factorization models that directly operate on user-item interaction matrices, NCF is an explicit model that learns user and item embedding vectors to predict ratings, allowing it to learn nonlinear user-item interaction relationships.

[0082] [2] Sequence recommendation models based on RNN / CNN:

[0083] GRU4Rec: This method uses a Gated Recurrent Unit (GRU) structure to capture long-term dependencies in user behavior sequences.

[0084] Caser: Combines the advantages of convolutional neural networks and recurrent neural networks, using one-dimensional convolutional layers to capture local temporal patterns and local dependencies in user behavior sequences.

[0085] [3] Sequence recommendation models based on attention mechanism / Transformer:

[0086] SASRec: Uses multi-head self-attention mechanism to capture sequence patterns of user historical behavior without repeated operations on input sequences.

[0087] BERT4Rec: A model based on the Transformer architecture, which uses the bidirectional encoder characteristics of BERT to consider both forward and backward dependencies in user behavior sequences, capturing richer context information.

[0088] MATN: This method designs a Transformer-based memory-enhanced network to model the interdependence of multiple behaviors.

[0089] [4] Sequence recommendation models based on graph neural networks:

[0090] SR-GNN: This model designs a gated graph neural network to process user session data.

[0091] HyperRec: Uses hypergraph to represent user behavior sequences to predict the next interaction item of users, and the hypergraph structure allows HyperRec to capture multi-dimensional relationships in user behavior sequences (co-occurrence relationships between items, similarity between users, and user behavior time series information, etc.)

[0092] [5] Multi-behavior recommendation models:

[0093] MAGNN: Uses a memory-augmented graph neural network to encode item long-term and short-term dependencies, and aggregates them through a fusion layer.

[0094] MBGCN: This model integrates user multi-behavior data into a unified graph to alleviate the problem of data sparsity, and then uses graph convolutional neural networks to learn node representations.

[0095] TGT: Designs a temporal graph encoder to jointly encode node representations with time and behavior context information.

[0096] MBSSL: A multi-behavior self-supervised learning framework and adaptive optimization method are designed to fuse time and behavior information for node self-discrimination at both inter-behavior and intra-behavior levels.

[0097] KEMB-Rec: User behavior patterns are mined using user behavior hypergraph, and semantic relationships are mined based on single-behavior interaction graph.

[0098] Tables 2 and 3 are the performance of these models and the CMB model on all indicators on the two data sets. From the tables, it can be seen that the performance of the patent model on the HR@N and NDCG@N indicators on the two data sets is better than that of other methods, and at least 5% performance improvement compared with the advanced multi-behavior recommendation method.

[0099] Table 2

[0100] HR@N and NDCG@N performance of all models on the Taobao dataset (N=1, 5, 10)

[0101]

[0102] Table 3

[0103] HR@N and NDCG@N performance of all models on the IJCAI dataset (N=1, 5, 10)

[0104]

[0105] The above describes the implementation process and advantages of the present application. Those skilled in the art should understand that without departing from the principles of the present application, various changes and improvements can be made to the present application, and these changes and improvements fall within the scope of the claimed present application.

Claims

1. A Transformer-based cross-sequence multi-behavior sequence recommendation method, characterized in that, Comprising the following steps: Step 1, segmenting user historical interaction sequences; Step 2, cross-sequence heterogeneous relation propagation; Step 3, short-term interaction pattern learning; Step 4, global representation aggregation; Step 5, generating a recommended content list by combining user recent preferences and item embeddings, while considering content priorities, by adjusting model parameters using user historical interactions for label data augmentation; According to the step 1, the user historical interaction sequence is segmented, and the method comprises the following steps: User set U = {u1,..., u i ,…,u I}, item set V = {v1,..., v j ,…v J}, a plurality of interaction behavior types defined as a set B contains all possible behavior types, where I, J ∈ Z represent the number of users and items, respectively; The historical interactions of each user are recorded as a multi-behavior interaction record, for a user u i ∈ U, whose historical interaction information is defined as a triple (v i,x , b i,x , t i,x ), representing the xth item of user u i interacts with b i,x type behavior at time x, constituting the multi-behavior interaction record of user u i , denoted as S i = {(v i,1 , b i,1 , t i,1 ), …, (v i,x , b i,x , t i,x ), …, (v i,X , b i,X , t i,X )}, where X represents the sequence length |S i |; The interaction sequence of a user is segmented by a Seg(·) operation to obtain a plurality of sub-sequences, for a given user u i and the interaction sequence S of the user u i The Seg(·) operation is expressed as: where P denotes the number of segmented user sub-sequences, b target denotes the target behavior, and denotes the u i the length of the first and last sub-sequence, denotes the maximum length of a sub-sequence; If the user has not performed the target behavior, then the maximum sub-sequence length segmenting a history interaction sequence of a user; Under the global view, if the last item in all user partitioned interaction sub-sequences is the same item that interacts with the target behavior, then the two sub-sequences are considered similar; given a user u i the kth sub-sequence of u user u i′ the sub-sequence of u if then the sequence is similar to the sequence denoted as Each sub-sequence has multiple similar sub-sequences; The cross-sequence heterogeneous relation propagation of step 2 is as follows: An embedding network is designed to jointly incorporate behavior type information and temporal signals as contextual information into the feature representation of items, the initial embedding e j ∈R d An item embedding matrix M ∈ R is generated by embedding the item set V into the same space |I|×d is obtained, where d represents the dimension of the latent state; e b ∈R d is associated with the user behavior type, representing the embedding of the b-th type of behavior; A time encoding strategy is introduced in the context information encoding layer, that is, a scalar timestamp is mapped to a vector space by using the formula: where τ(·) denotes a function that maps a timestamp to a 2d-dimensional vector, W t ∈ R 2d×d denotes a time transform matrix, the odd and even indices of the 2d-dimensional vector are denoted by 2l and 2l+1; using as a scaling factor, W t the matrix introduces an adjustable linear projection mechanism; The behavior type representation and time context information are jointly injected into the item representation by the following operation: wherein denotes an item embedding infused with contextual information as input to the cross-sequence information passing component, denotes a vector element-wise addition operation; A cross-sequence information propagation encoder based on the Transformer architecture and multi-head representation space attention mechanism is designed; The embedding of the xth interaction item in the sequence is calculated by the multi-head dot product attention method as shown in the formula: wherein denotes the user u i the refined representation of the x-th interaction item in the k-th sub-sequence, Y denotes the item set of related items, E y denotes the embedding representation of the related items, a x,y denotes the learned correlation coefficient between the item and v y , and σ(·) denotes the softmax function, h e H denotes the h-th head of the attention layer, denotes the Query, Key and Value transformation matrices of the h-th head, respectively, and the score of the query with the value vector is divided by to stabilize the gradient; The short-term interaction pattern learning of step 3 is as follows: A behavior-aware encoder is designed to aggregate specific types of behavior patterns in short-term; for a given user u i with sequence u i The representation of the b-th type of behavior pattern in the corresponding short-term is denoted as: where denotes the aggregated computed u i In characterizes the behavior of b at time period, and p(·) is a defined judgment function whose result is 1 if the two inputs are equal, otherwise 0, is the item refined embedding representation output by the cross-sequence heterogeneous relation propagation module, W b is the behavior-aware transformation matrix used to enhance the semantics of specific types of behaviors, whose optimization path is based on the multi-channel parameter learning strategy shown in the formula: where M denotes the number of channels, P e R M×d and μ e R M denote the transformation matrix and bias parameters in the fully connected layer, respectively, denotes the transformation matrix corresponding to the m-th channel. After obtaining the embedding representation of the specific behavior type, the short-term behavior type code needs to be fed into the aggregation layer to capture the potential interdependence between different types of behaviors and obtain the user's short-term feature representation; an adaptive attention network is used to calculate the contribution index of each behavior type to the short-term representation, and the aggregation layer operation is formalized as: where is the user u i the feature representation of the kth sub-sequence, i.e. u i the short-term preference encoding over the corresponding period, β i,b denotes the embedding of the sum of all actions as Query learned by u i the attention weight for the corresponding b behavior, W A and μ A are the transformation matrix and bias of the aggregation layer, respectively; For the message passing process from the user to the item side, the embedding code of the item is also subjected to the propagation and aggregation process described above: wherein and E j respectively represent the items v j a particular behavior type embedding and a fused behavior perception embedding; The global representation aggregation of step 4 is as follows: A global graph relational encoder is designed to inject the embedding representation of each period of users into the global feature representation, capturing the multi-behavior dynamic pattern of users from a long-term and short-term perspective; a global user graph G is defined to contain all user nodes u i ∈U and their interaction subsequence nodes That is, G = {U, S, ε}, where S and ε represent the set of all user subsequence and the edge connecting each user u i and his interaction subsequence nodes The global representation aggregation module performs graph structure information aggregation on the global user graph G to generate the global embedding representation of the user, and integrates the temporal context information in the information aggregation process, so that the short-term representations of different periods interact in a distinguishable way. The calculation process is: where is the global feature representation of user u i , σ(·) is the softmax activation function, and η k is the learned attention weight of user u i to its kth sub-sequence, is the time encoding returned by the context information encoding layer, corresponding to the interaction occurrence time of the last item in the sequence . The model performs information aggregation and encodes the user's short-term interaction behavior pattern and long-term dynamic user embedding representation. During the embedding propagation process, the model uses the user-item interaction graph and the global user graph G as the computation graph, and during this process, the local relationship features from the neighborhood will be aggregated to obtain the context representation. Such a message passing paradigm is summarized by merging high-order connectivity as: wherein and are item and user embeddings, respectively, for encoding layer initialization; by extending the L-layer graph relational encoder, the ability of the model to capture high-order neighbor collaborative signals is improved; by corresponding addition operation of L-layer embedding elements, the final user item and subsequence feature representation, the process of which is formalized as: The step 5 of generating a recommended content list by combining user recent preferences and item embeddings, while considering content priorities, by adjusting model parameters using user historical interactions for label data augmentation, is as follows: In the prediction layer of the model, the last stage short-term feature representation of the user is jointly used with the item feature representation E j To predict the possible future interaction items of the user u i and the item v j The possibility of the upcoming interaction is quantified as where is a parameter vector, denotes the element-wise multiplication operation, and in the training phase, the parameter adjustment process is optimized by using user historical interactions for label data augmentation, so that u i and v j In the prediction score in the corresponding period is calculated as: The objective function of the model training is formalized as marginal pairwise loss. where I and K i respectively represent the number of users and the user u i , C represents the number of sampling, p c and n c respectively represent the corresponding positive and negative samples, and further introduce a weight decay regularization term to alleviate the overfitting phenomenon.

Images