Personalized recommendation method for multi-behavior association modeling and interest boundary perception

Abstract

The invention provides a personalized recommendation method for multi-behavior association modeling and interest boundary perception. Comprising a transformer-based multi-behavior sequence modeling module, a time-aware graph aggregation module and a user interest boundary prediction module, and a complete system is realized through training. An interaction sequence generated by a user before the current moment and a corresponding behavior type are embedded and then pass through the transformer-based multi-behavior sequence modeling module to obtain a behavior and sequence sensing node representation, and then through the time sensing graph aggregation module, a representation of a one-hop center node is obtained through aggregation according to the distance of a time interval. Stacking L layers to expand the user or project interaction diagram to L hops to obtain user node and project node representations; and finally, performing loss function optimization through the user interest boundary prediction module after the user node and the project node representation are subjected to MLP to obtain a prediction result of the user interest, and realizing personalized recommendation.

Description

Technical Field

[0001] The present disclosure relates to the field of information retrieval, and more specifically, to a personalized recommendation method based on multi-behavior association modeling and interest boundary perception. Background Art

[0002] Recommendation systems are widely used in various online platforms and services. Their purpose is to find the items that best match the user's interests based on the user's personal preferences. With the exponential growth of online information, efficient and accurate personalized recommendation systems have become increasingly important. In a typical recommendation scenario, the recommendation model calculates the matching score between the user and the favorite items based on the user's historical data and interaction behavior, and selects a list of items to be recommended to the user based on the matching score. By improving the personalization and accuracy of the recommendation model algorithm, the items that the user may be interested in will be more exposed, which will help increase the platform's sales volume. Past algorithms mainly include single-behavior sequence recommendation algorithms, multi-behavior recommendation algorithms, and multi-behavior sequence recommendation algorithms.

[0003] The existing single-behavior sequence recommendation algorithms use historical behavior sequences composed of a single type of feedback to model users' long-term and short-term preferences to predict items that may interact in the future. However, in real scenarios, there is usually more than one type of interaction between users and items, such as browsing, liking, adding to shopping carts, and purchasing. Considering the heterogeneity of user feedback, researchers have turned to the study of multi-behavior recommendation algorithms in recent years. By modeling the dependencies between heterogeneous behaviors, the recommendation system is closer to the user's real behavior, and the recommended products are more in line with the user's real preferences. Some early multi-behavior recommendation algorithms used matrix decomposition methods to model the dependencies between user behaviors. With the rise of graph neural networks and transformers, more models use these two architectures to model multi-behavior dependencies in user interaction sequences.

[0004] In addition, in order to simultaneously model users' heterogeneous behaviors and sequential information of behaviors, researchers proposed a multi-behavior sequence recommendation model. The existing multi-behavior sequence recommendation model injects behavior and sequential information into items in user interaction sequences by modeling heterogeneous graph neural networks or utilizing the self-attention mechanism in transformers to capture short-term and long-term cross-type behavior dependencies.

[0005] Single-behavior sequence recommendation and multi-behavior recommendation ignore the heterogeneity and sequentiality of user behavior respectively. They cannot model the above two properties of user behavior at the same time, so there is still a certain gap with the real user interaction scenario. Single-behavior sequence recommendation is prone to data sparsity or cold start problems. Although multi-behavior recommendation takes into account the heterogeneity of behavior, ignoring the sequential information will result in the loss of contextual information of user interaction, which will affect the recommendation effect.

[0006] Existing models have explored the modeling of user preferences through user-item interaction graphs and sequential interactions using advanced techniques such as graph neural networks and transformer-based architectures. However, these methods are generally insufficient in simultaneously considering the dynamic characteristics of graph topology and the sequential patterns of interactions in user preference models. Existing models mainly capture the interests of users within their own sequences, while ignoring the high-order correlations between different users or items, such as similar preferences between users or similar attributes between items, resulting in the need for longer interaction sequences to capture user preferences. At the same time, existing methods model static user interaction sequences, making it difficult to capture user preferences at intermediate moments. Second, some methods use positional encoding to model sequences, ignoring the time intervals between different positions, making it difficult to capture the dynamic changes of user interests in a timely manner and possibly introducing noise in recommendations. Finally, in terms of model personalization, different interest boundaries of different users often affect the effectiveness of recommendations. Summary of the invention

[0007] The purpose of the embodiments of the present disclosure is to provide a personalized recommendation method based on multi-behavior association modeling and interest boundary perception.

[0008] In general, a personalized recommendation method based on multi-behavior association modeling and interest boundary awareness is provided, which consists of three parts: a transformer-based multi-behavior sequence modeling module, a time-aware graph aggregation module, and a user interest boundary prediction module, and a complete system is realized through training;

[0009] The interaction sequence and corresponding behavior type generated by the user before the current moment are embedded and then passed through the transformer-based multi-behavior sequence modeling module to obtain the behavior and order-aware node representation. Then, the time-aware graph aggregation module is used to aggregate the time intervals to obtain the representation of the 1-hop central node. The user or project interaction graph is expanded to L hops by stacking L layers to obtain the user node and project node representation. Finally, the user node and project node representations are passed through the MLP and the loss function is optimized through the user interest boundary prediction module to obtain the prediction result of user interest and realize personalized recommendation.

[0010] The specific process of embedding is to embed the user u, the item sequence V interacting with the user u ={v1,...,v n}、Corresponding interactive behavior sequence B u ={b1,…,b n} and the timestamp sequence T of the interaction occurrence u ={t1,...,t n} as input and perform the following prediction task: n+1 User u and item v at timen+1 Generate the probability of target behavior interaction and recommend high-probability items to users;

[0011] The transformer-based multi-behavior sequence modeling module is implemented as follows: the initial representation E(v) after the neighbor item sequence is embedded is E(v1),...,E(v2). n )} as input, the dependencies between multiple behaviors are captured in the neighbor interaction sequence through multi-behavior multi-head self-attention, and the behavior order information is injected into the neighbor interaction sequence through multi-behavior relative position encoding to explore the functional relationship between behaviors at different positions. Finally, the neighbor sequence representation E containing heterogeneous behaviors and order information is obtained. l (v) = {E l (v1),...,E l (v n )};

[0012] The time-aware graph aggregation module first uses time-aware behavior encoding to assign time and behavior information to the sequence representation of neighboring nodes, namely, the neighbor sequence representation E l (v) Then get the node representation z of injected timing and behavior information i (t); Then, the time span and behavior-aware sequence node representations are aggregated through the time graph attention mechanism to obtain the final target node representation

[0013] The specific method of capturing the dependency between multiple behaviors in the neighbor interaction sequence through multi-behavior multi-head self-attention is as follows: the initial representation E(v) of the input neighbor item sequence is first subjected to a behavior-specific linear mapping to perform q, k, v decomposition:

[0014] {q,k,v}={f q ,f k ,f v}(E(v),b)

[0015] where f q ,f k ,f v It is a linear transformation method that linearly maps the original input tensor dimension to the model multi-head dimension through a linear transformation matrix. Unlike the original transformer that directly multiplies the input tensor with the linear transformation matrix, we introduce the one-hot encoding of the behavior sequence b, and multiply the input tensor, the linear transformation matrix, and the one-hot encoding of the behavior b to complete the behavior-specific q, k, v linear mapping.

[0016] Then, when calculating the original attention score, the behavior attention weight matrix is ​​introduced to capture the different effects of other position behaviors on the current behavior, and the relative position encoding is introduced to add order information to the sequence:

[0017]

[0018] Where i and j represent the positions in the sequence, and q[i] and k[j] are the decomposed q and k tensors at the corresponding positions. Different from the direct multiplication of q and k in the traditional transformer, we construct a different parameter matrix for each behavior pair. Captures the multi-behavior dependencies between different positions. d is the scaling factor in the dot product attention mechanism to prevent the dot product value from being too large, which is the same as the dimension of the query vector q. P[i,j] is the multi-behavior position encoding, which injects behavior order information into the sequence. After adding and passing Softmax, the final raw attention score A[i,j] between positions i and j is obtained.

[0019] Finally, the final attention score is calculated by multi-head concatenation, and then passes through the MLP with specific behavior, layer normalization and residual connection:

[0020]

[0021] S l [i]=Concat(S1[i],...,S h [i])

[0022] E l (v i )=MLPs(S l [i],b)

[0023] S l [i] = LayerNorm(E(v)+S l (i))

[0024] E l (v) = LayerNorm(S l (i)+E l (v))

[0025] Similar to calculating the original attention score, in order to get the value vector v of the heterogeneity of behaviors from different positions j In order to aggregate information, alleviate the distribution differences between different behaviors and calculate the final attention output S[i], we construct a Parameter matrices of the same dimension Capture the heterogeneous relationship between the behaviors of each pair during aggregation. Output a total of h multi-heads S1[i],...,S h [i] After concatenation, the multi-head representation S is obtained. l [i]. Then, the long position is characterized by S according to the behavior type b at position i.l [i] The multi-head representation dimensions are integrated into the dimensions required by the final model by sending them to the MLP of the corresponding behavior, thereby obtaining the final neighbor sequence representation E containing behavioral heterogeneity and sequential information. l (v) LayerNorm is a commonly used regularization method. The core idea is to perform normalization on the feature dimension of each sample so that the output of each layer has the same distribution, thereby avoiding the adverse effects of some feature values ​​being too large or too small.

[0026] Thus, the final neighbor sequence representation E containing behavioral heterogeneity and order information is obtained l (v).

[0027] The specific implementation method of the multi-behavior relative position coding to inject behavior order information into the neighbor interaction sequence to explore the functional relationship between behaviors at different positions is:

[0028]

[0029] Each encoding function Corresponding to a behavior type pair, it encodes the order relationship between the two behavior types. In particular, for any two behaviors indexed by i and j in the sequence, ji is the relative distance between j and i. () is the behavior type pair (b i ,b j ) The specific meaning of the encoding function can be expressed as follows: when the relative distance is close, an independent position is assigned to each position. As the relative distance increases, multiple positions share one position until the relative distance reaches a threshold and is truncated. The specific segmentation processing formula is expressed as:

[0030]

[0031] Where i and j are the indices of a pair of behaviors in the behavior sequence, ji is the relative distance between j and i, and n b is the number of position codes after segmentation, that is, the total number of independent positions and shared positions, and n is the maximum relative distance, that is, the length of the sequence.

[0032] The time-aware behavior encoding endows the sequence representation with time and behavior information, and introduces a continuous time mapping function for the time information:

[0033]

[0034] where φ(t) is the temporal encoding function, d is the dimension of the temporal encoding, ω1,...,ω d It is an independent and identically distributed random variable from the probability density function distribution P(ω), and t is a specific timestamp.

[0035] As the time representation of the node, in the multiplication of the self-attention mechanism, the above time function satisfies:

[0036] φ(t1) T φ(t2)≈φ(t1-t2)=K(t1,t2)

[0037] Where φ(t1) T refers to the transpose of the time function at time t1, φ(t2) refers to the time function at time t2, φ(t1) T Multiply it with φ(t2) in the self-attention mechanism. t1-t2 represents the time difference between the two positions, and K() is a defined time kernel function. Through the above equation, the product of the time functions of the two moments reflects the time span information, which can well capture the time span information in the multiplication calculation of the self-attention mechanism; at the same time, in order to add the behavior information of the interactive edge to the representation of the neighboring node, the behavior is mapped and spliced ​​into the node representation. Finally, the node after time-aware behavior encoding can be expressed as:

[0038] z i (t) = [E l (v i )||x 0,i (t i )||φ(tt i )]

[0039] Where E l (v i ) is the node v i The node representation obtained after passing through l layers of multi-behavior transformers, x 0,i (t i ) is the target node 0 and node i at t i The interactive behavior type generated at each moment is mapped and embedded, φ(tt i ) contains the time span information of the interaction between nodes 0 and i. i (t) is the node v i The representation after fusing behavior and time information at time t.

[0040] The time-series graph attention layer completes the representation aggregation operation of neighbor nodes, and uses the self-attention mechanism to represent the sequence nodes of the previously obtained time-series and behavior perception:

[0041]

[0042] Decompose, Z(t) or E l (v1)||x 0,1 (t1)||φ(0),...,E l (v n )||x0,n (t n )||φ(tt n )] represent the neighbor nodes v1,...,v n The initial representation of the neighbor node representation after fusing the behavior representation and time information, x 0,n (t n ) refers to n The mapping representation of the behavior type generated between node 0 and node n at time φ(tt n ) is the time encoding function about time t:

[0043] q(t)=[z0(t)]W Q ,

[0044] K(t)=[Z(t)] 1:n W K ,

[0045] V(t)=[Z(t)] 1:n W V

[0046] Where z0(t) is the representation of the central node 0 after integrating the behavior and time information, [Z(t)] 1:n are neighbor nodes v1,...,v n The representation after integrating behavior and time information, W Q ,W K ,W V It corresponds to the linear transformation matrix, through W Q ,W K ,W V The linear transformation matrix of is multiplied to obtain the decomposed and time-aware query vector q(t), key vector K(t) and value vector V(t). And the final attention score is calculated:

[0047]

[0048] q(t) is the time-aware query vector, K(t) T is the transpose of the time-aware key vector, d has the same dimension as the query vector q(t), V(t) is the time-aware value vector, and h0(t) is the final time-aware representation of the central node 0 after aggregating its neighbor nodes.

[0049] Finally, it is extended to a multi-head self-attention mechanism, through residual connections and layer normalization, and the final target node embedding is expressed as:

[0050]

[0051] in They are the time-aware representations of node 0 after aggregation on h heads, || represents concatenation, and z0 is the initial representation of node 0 itself. FFN() is a multi-layer neural network for feature transformation. LN refers to LayerNorm, which is a commonly used regularization method. The core idea is to normalize the feature dimension of each sample so that the output of each layer has the same distribution, thereby avoiding the adverse effects of some feature values ​​being too large or too small. It is the final representation of time perception after the time-aware target node 0 aggregates all neighbor nodes before the current moment, and (L+1) means that the current aggregation occurs in the (L+1) hop of the graph.

[0052] The user interest boundary prediction module introduces an auxiliary score b for each user's interaction behavior at timestamp t u,k (t), which represents the user interest boundary under different behaviors:

[0053]

[0054] Where u(t) is the learned user representation at time t, is the representation of the learned behavior edge at time t, It is the multi-behavior relationship matrix learned earlier.

[0055] The loss function of the training process is in point-by-point form, and the scores between user u and the positive term p and the negative term n are expressed as:

[0056]

[0057] in represents the representation of user node u after (L+1) hop aggregation at the current time t, represents the representation of the positive item node p after (L+1) hop aggregation at the current time t, It represents the representation of the negative item node n after (L+1) hop aggregation at the current time t. FFN is a multi-layer neural network for dimensionality conversion. s(u,p) refers to the score between the user and the positive item, and s(u,n) refers to the score between the user and the negative item.

[0058] The corresponding loss function is expressed as:

[0059]

[0060] Where σ refers to the sigmoid() function, and the loss function penalizes negative samples with scores above the boundary and positive samples with scores below the boundary, b u,k(t) An interest boundary is established for each user behavior interaction, providing different boundaries for different user behaviors; the optimal score distribution is found by adjusting the hyperparameter α to balance the score distribution.

[0061] The innovation of the embodiment of the present invention lies in:

[0062] (1) We propose a new framework for multi-behavior sequential recommendation, which reveals the dynamic changes of user preferences and the multi-behavior dependencies between user-item interactions.

[0063] (2) Through the transformer-based multi-behavior association module, behavior-aware sequence modeling is performed, time graph aggregation is used to capture users' intermediate preferences, and the multi-hop graph structure captures the high-order associations between users and items.

[0064] (3) A learnable explicit score is introduced into the loss function to represent the user's interest boundary between different behaviors. The recommendation effect is further improved by adjusting the boundary distribution to achieve personalized recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other objects and features of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings.

[0066] Figure 1 It is a schematic diagram showing the architecture diagram of a dynamic graph-based multi-behavior sequence recommendation method based on interest boundary awareness according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0068] The features described herein can be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, which will be clear after understanding the disclosure of the present application.

[0069] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more.

[0070] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Therefore, without departing from the teachings of the examples described herein, the first member, first component, first region, first layer, or first portion referred to in the examples may also be referred to as the second member, second component, second region, second layer, or second portion.

[0071] In the specification, when an element (such as a layer, a region or a substrate) is described as being “on”, “connected to” or “coupled to” another element, the element may be directly “on”, “connected to” or “coupled to” another element, or one or more other elements may be present therebetween. Conversely, when an element is described as being “directly on”, “directly connected to” or “directly coupled to” another element, there may be no other elements present therebetween.

[0072] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0073] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by a person of ordinary skill in the art to which the present disclosure belongs after understanding the present disclosure. Unless explicitly defined as such herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and should not be interpreted in an idealized or overly formal manner.

[0074] Furthermore, in the description of examples, when it is considered that a detailed description of a well-known related structure or function would cause vague interpretation of the present disclosure, such a detailed description will be omitted.

[0075] Figure 1 It is a schematic diagram showing a personalized recommendation method of multi-behavior association modeling and interest boundary perception according to an embodiment of the present disclosure.

[0076] This scheme proposes an interest boundary-aware dynamic graph transformer algorithm MB-DGT for multi-behavior sequence recommendation, which takes into account the dynamics, heterogeneity and sequentiality of user interaction sequences in recommendation scenarios. The architecture of our model MB-DGT is shown in the figure below. The model mainly consists of three parts: a transformer-based multi-behavior sequence modeling module, a time-aware graph aggregation module and a user interest boundary prediction module. After the user's interaction sequence and the corresponding behavior type are embedded, the behavior and sequence-aware node representation is obtained through the transformer-based multi-behavior sequence modeling module. Then, the time-aware graph aggregation module is used to aggregate the time interval to obtain the representation of the 1-hop central node. Stacking L layers can expand the user or project interaction graph to L hops. Finally, the user node and project node representations are passed through the MLP and the loss function is optimized through the user interest boundary prediction module to achieve personalized recommendations.

[0077] The MB-DGT model aims to recommend items that the user may be interested in at the next moment based on the historical interaction records of multiple user behaviors. In order to more comprehensively extract user preferences, MB-DGT combines user u, the item sequence V with which the user interacts, and u ={v1,...,v n}、Corresponding interactive behavior sequence B u ={b1,…,b n} and the timestamp sequence T of the interaction occurrence u ={t1,...,t n} as input and perform the following prediction task: n+1 User u and item v at time n+1 Generate the probability of target behavior interaction and recommend high-probability items to users.

[0078] Transformer's multi-behavior sequence modeling module

[0079] Use multi-behavior multi-head self-attention to capture the dependencies between behaviors, and inject sequence signals into the behavior sequence through multi-behavior relative position encoding. Take the initial representation E(v) = {E(v1),...,E(v n )} as input, the goal is to obtain a neighbor sequence representation E containing behavioral heterogeneity and sequential information l (v) = {E l (v1),...,E l (v n )}.

[0080] The goal of multi-behavior multi-head self-attention is to capture the dependencies between multiple behaviors in the neighbor interaction sequence. Referring to the transformer architecture, the initial representation E(v) of the input neighbor item sequence is first subjected to a behavior-specific linear mapping to perform q, k, v decomposition:

[0081] {q,k,v}={f q ,f k ,f v}(E(v),b)

[0082] where f q ,f k ,f v It is a linear transformation method that linearly maps the original input tensor dimension to the model multi-head dimension through a linear transformation matrix. Unlike the original transformer that directly multiplies the input tensor with the linear transformation matrix, we introduce the one-hot encoding of the behavior sequence b, and multiply the input tensor, the linear transformation matrix, and the one-hot encoding of the behavior b to complete the behavior-specific q, k, v linear mapping.

[0083] Then, when calculating the original attention score, the behavior attention weight matrix is ​​introduced to capture the different effects of other position behaviors on the current behavior, and the relative position encoding is introduced to add order information to the sequence:

[0084]

[0085] Where i and j represent the positions in the sequence, and q[i] and k[j] are the decomposed q and k tensors at the corresponding positions. Different from the direct multiplication of q and k in the traditional transformer, we construct a different parameter matrix for each behavior pair. Captures the multi-behavior dependencies between different positions. d is the scaling factor in the dot product attention mechanism to prevent the dot product value from being too large, which is the same as the dimension of the query vector q. P[i,j] is the multi-behavior position encoding, which injects behavior order information into the sequence. After adding and passing Softmax, the final raw attention score A[i,j] between positions i and j is obtained.

[0086] Finally, the final attention score is calculated and concatenated, and then the MLP with specific behavior, layer normalization and residual connection are performed to obtain the final neighbor sequence representation E containing behavior heterogeneity and sequential information. l (v).

[0087]

[0088] S l [i]=Concat(S1[i],...,S h [i])

[0089] E l (v i )=MLPs(S l [i],b)

[0090] S l [i] = LayerNorm(E(v)+S l (i))

[0091] E l (v) = LayerNorm(S l (i)+E l (v))

[0092] Similar to calculating the original attention score, in order to get the value vector v of the heterogeneity of behaviors from different positions j In order to aggregate information, alleviate the distribution differences between different behaviors and calculate the final attention output S[i], we construct a Parameter matrices of the same dimension Capture the heterogeneous relationship between the behaviors of each pair during aggregation. Output a total of h multi-heads S1[i],...,S h [i] is concatenated, i.e., the Concat operation is performed to obtain the multi-head representation Sl[i]. Then, the multi-head representation S is concatenated according to the behavior type b at position i. l [i] The multi-head representation dimensions are integrated into the dimensions required by the final model by sending them to the MLP of the corresponding behavior, thereby obtaining the final neighbor sequence representation E containing behavioral heterogeneity and sequential information. l (v) LayerNorm is a commonly used regularization method. The core idea is to perform normalization on the feature dimension of each sample so that the output of each layer has the same distribution, thereby avoiding the adverse effects of some feature values ​​being too large or too small.

[0093] The goal of multi-behavior relative position encoding is to inject behavior order information into the neighbor interaction sequence to explore the relationship between behaviors at different positions:

[0094]

[0095] Each encoding function Corresponding to a behavior type pair, it encodes the order relationship between the two behavior types. In particular, for any two behaviors indexed by i and j in the sequence, ji is the relative distance between j and i. () is the behavior type pair (b i ,b j) When choosing the position encoding method, it is considered that directly mapping relative positions to real numbers will lead to uneven learning, especially for long relative positions. For example, in a sequence of length 30, there are 29 pairs of positions with a relative distance of 1, while there is only 1 pair of positions with a relative position of 29. Therefore, referring to the T5 model, when the relative distance is close, an independent position is assigned to each position. As the relative distance increases, multiple positions share a position until the relative distance reaches a threshold and is truncated. This method effectively balances the number of short-distance position pairs and allows for more balanced modeling of long-length behavior sequences. The specific segmentation processing formula is expressed as:

[0096]

[0097] Where i and j are the indices of a pair of behaviors in the behavior sequence, ji is the relative distance between j and i, and n b is the number of position codes after segmentation, that is, the total number of independent positions and shared positions, and n is the maximum relative distance, that is, the length of the sequence.

[0098] Time-aware graph aggregation module

[0099] The purpose of the time-aware graph aggregation module is to complete the time-aware aggregation of target nodes using the previously obtained neighbor sequence representation. First, we use time-aware behavior encoding to assign time and behavior information to the sequence representation of neighbor nodes, that is, the neighbor sequence representation E obtained previously l (v) Then get the node representation z of injected timing and behavior information i (t). Then, the time span and behavior-aware sequence node representations are aggregated through the time graph attention mechanism to obtain the final target node representation.

[0100] The goal of time-aware action encoding is to endow sequence representation with temporal and behavioral information. For temporal information, we introduce a continuous time mapping function:

[0101]

[0102] where φ(t) is the temporal encoding function, d is the dimension of the temporal encoding, ω1,...,ω d It is an independent and identically distributed random variable from the probability density function distribution P(ω), and t is a specific timestamp.

[0103] As the time representation of the node, in the multiplication of the self-attention mechanism, the above time function satisfies:

[0104] φ(t1) T φ(t2)≈φ(t1-t2)=K(t1,t2)

[0105] Where φ(t1) T refers to the transpose of the time function at time t1, φ(t2) refers to the time function at time t2, φ(t1) T Multiply it with φ(t2) in the self-attention mechanism. t1-t2 represents the time difference between the two positions, and K() is a defined time kernel function. Through the above equation, the product of the time functions of the two moments reflects the time span information, which can well capture the time span information in the multiplication calculation of the self-attention mechanism. At the same time, in order to add the behavior information of the interaction edge to the representation of the neighboring node, the behavior is mapped and spliced ​​into the node representation. Finally, the node after time-aware behavior encoding can be expressed as:

[0106] z i (t) = [E l (v i )||x 0,i (t i )||φ(tt i )]

[0107] Where E l (v i ) is the node v i The node representation obtained after passing through l layers of multi-behavior transformers, x 0,i (t i ) is the target node 0 and node i at t i The interactive behavior type generated at each moment is mapped and embedded, φ(tt i ) contains the time span information of the interaction between nodes 0 and i. i (t) is the node v i The representation after fusing behavior and time information at time t.

[0108] The goal of the time-series graph attention layer is to complete the aggregation operation of the representation of neighbor nodes. The self-attention mechanism is used to represent the sequence nodes of the time and behavior perception obtained above:

[0109]

[0110] Decompose, Z(t) or E l (v1)||x 0,1 (t1)||φ(0),...,E l (v n )||x 0,n (t n )||φ(tt n )] represent the neighbor nodes v1,...,v n The initial representation of the neighbor node representation after fusing the behavior representation and time information, x 0,n (tn ) refers to n The mapping representation of the behavior type generated between node 0 and node n at time φ(tt n ) is the time encoding function about time t:

[0111] q(t)=[z0(t)]W Q ,

[0112] K(t)=[Z(t)] 1:n W K ,

[0113] V(t)=[Z(t)] 1:n W V

[0114] Where z0(t) is the representation of the central node 0 after integrating the behavior and time information, [Z(t)] 1:n are neighbor nodes v1,...,v n The representation after integrating behavior and time information, W Q ,W K ,W V It corresponds to the linear transformation matrix, through W Q ,W K ,W V The linear transformation matrix of is multiplied to obtain the decomposed and time-aware query vector q(t), key vector K(t) and value vector V(t). And the final attention score is calculated:

[0115]

[0116] q(t) is the time-aware query vector, K(t) T is the transpose of the time-aware key vector, d has the same dimension as the query vector q(t), V(t) is the time-aware value vector, and h0(t) is the final time-aware representation of the central node 0 after aggregating its neighbor nodes.

[0117] Finally, extended to the multi-head self-attention mechanism, through residual connection and layer normalization, the final target node embedding can be expressed as:

[0118]

[0119] in They are the time-aware representations of node 0 after aggregation on h heads, || represents concatenation, and z0 is the initial representation of node 0 itself. FFN() is a multi-layer neural network for feature transformation. LN refers to LayerNorm, which is a commonly used regularization method. The core idea is to normalize the feature dimension of each sample so that the output of each layer has the same distribution, thereby avoiding the adverse effects of some feature values ​​being too large or too small. It is the final representation of time perception after the time-aware target node 0 aggregates all neighbor nodes before the current moment, and (L+1) means that the current aggregation occurs in the (L+1) hop of the graph.

[0120] User interest boundary prediction module

[0121] The purpose of the user interest boundary prediction module is to take into account that different user behaviors usually have different interest boundaries. In order to effectively model the personalized behavior boundaries of users and consider the dependencies between behaviors, we introduce an auxiliary score b for each user's interaction behavior at timestamp t. u,k (t), which represents the user interest boundary under different behaviors. It can be a learned parameter or a parameter calculated from the user and edge embedding behavior information:

[0122]

[0123] Where u(t) is the learned user representation at time t, is the representation of the learned behavior edge at time t, It is the multi-behavior relationship matrix learned earlier.

[0124] Training and Optimization

[0125] The loss function of MB-DGT adopts a point-by-point form, and the scores between user u and the positive term p and the negative term n can be expressed as:

[0126]

[0127] in represents the representation of user node u after (L+1) hop aggregation at the current time t, represents the representation of the positive item node p after (L+1) hop aggregation at the current time t, It represents the representation of the negative item node n after (L+1) hop aggregation at the current time t. FFN is a multi-layer neural network for dimensionality conversion. s(u,p) refers to the score between the user and the positive item, and s(u,n) refers to the score between the user and the negative item.

[0128] The corresponding loss function can be expressed as:

[0129]

[0130] Where σ refers to the sigmoid() function. In this loss function, our goal is to maximize the score of the positive example minus the boundary score and minimize the score of the negative example minus the boundary score. The loss function penalizes negative samples with scores above the boundary and positive samples with scores below the boundary. At the same time, b u,k (t) An interest boundary is established for each user behavior interaction, providing different boundaries for different user behaviors. The optimal score distribution can be found by adjusting the hyperparameter α to balance the score distribution.

[0131] Beneficial effects brought by the technical solution of the present invention

[0132] In order to verify the effectiveness of the model, we constructed some experiments for verification. Yelp, Taobao and Tmall data sets were selected for experimental verification. In order to better evaluate the experimental results, Hit, NDCG@5 and @10 were selected as evaluation indicators. We adopted the same evaluation method and training test set division method as the multi-behavior sequence recommendation work MB-STR, using the leave-one-out strategy. In particular, the final interaction representing the target behavior in each sequence is put into the test set as a positive example, and the remaining interactions constitute the training set. According to the evaluation method of the above method, 99 items are randomly selected from the entire project set as negative samples, and ranked and evaluated together with the positive samples.

[0133] Some benchmark models are selected for performance comparison, which can be divided into four benchmark methods: traditional recommendation methods such as DMF and NGCF that use matrix decomposition or collaborative filtering methods to explore collaborative signals between users; sequential recommendation algorithms that use users' historical sequence data to capture short-term and long-term preferences and predict users' future interactions, such as GRU4REC, Caser, SASRec, and BERT4Rec; multi-behavior recommendation algorithms that use users and items as embedding vectors and make recommendations based on previous multi-behavior interactions, such as NMTR, MB-GCN, and MB-GMN; multi-behavior sequence recommendation algorithms that simultaneously capture users' heterogeneous behaviors and sequential signals in behavior sequences, such as DIPN, BINN, MGNN-SPred, DMT, MBHT, and MB-STR algorithms.

[0134] The experimental results of various algorithms are shown in the following table:

[0135]

[0136] From the above experimental results, we can see that: (1) MB-DGT shows its effectiveness in the multi-behavior sequential recommendation problem. Compared with all baseline methods, the model provides superior performance on each metric and all datasets, which can be attributed to: 1) MB-DGT captures multi-behavior dependencies in the sequence through transformer-based multi-behavior correlation modeling, thereby better identifying the importance of items between different behaviors. 2) MB-DGT effectively captures the dynamic changes of user interests within the sequence and the high-order connectivity between sequences by constructing a 2-hop multi-behavior user-item temporal interaction graph and applying time-aware aggregation. 3) By learning the interest boundaries of user behaviors, MBDGT can make more accurate recommendations for target behaviors. (2) Both sequential and multi-behavior methods improve the performance of the model. Capturing sequential dependencies can improve the effectiveness of recommendations. In general, the sequential recommendation baseline outperforms most traditional baselines that do not utilize sequential information. The performance comparison of multi-behavior sequential recommendation and multi-behavior recommendation also proves the effectiveness of incorporating sequential information. In addition, the use of multi-behavior information enhances the modeling of user preferences. Multi-behavior recommendation generally outperforms traditional recommendation methods, while multi-behavior sequential recommendation generally improves prediction performance more than sequential recommendation methods. (3) MB-DGT consistently outperforms methods based on multi-behavior sequential recommendation. First, compared with RNN-based (such as DIPN and BINN) and GNN-based (such as MGNN-SPred) models, MB-DGT fully exploits transformer-based multi-behavior association modeling to capture short-term and long-term multi-behavior sequential dependencies. Second, compared with transformer-based models such as DMT, MBHT, and MB-STR, MB-DGT conveys higher-order connectivity through the multi-hop multi-behavior interaction graph of user-items and effectively models dynamic user preferences.

[0137] Although some embodiments of the present disclosure have been shown and described, it will be appreciated by those skilled in the art that modifications may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A personalized recommendation method based on multi-behavior association modeling and interest boundary perception, characterized in that: It consists of three parts: a transformer-based multi-behavior sequence modeling module, a time-aware graph aggregation module, and a user interest boundary prediction module, and a complete system is realized through training; The interaction sequence and corresponding behavior type generated by the user before the current moment are embedded and then passed through the transformer-based multi-behavior sequence modeling module to obtain the behavior and order-aware node representation. Then, the time-aware graph aggregation module is used to aggregate the time intervals to obtain the representation of the 1-hop central node. The user or project interaction graph is expanded to L hops by stacking L layers to obtain the user node and project node representation. Finally, the user node and project node representations are passed through the MLP and the loss function is optimized through the user interest boundary prediction module to obtain the prediction result of user interest and realize personalized recommendation. The specific process of embedding is to embed the user u, the item sequence V interacting with the user u ={v1,...,v n }、Corresponding interactive behavior sequence B u ={b1,…,b n } and the timestamp sequence T of the interaction occurrence u ={t1,...,t n } as input and perform the following prediction task: n+1 User u and item v at time n+1 Generate the probability of target behavior interaction and recommend high-probability items to users; The transformer-based multi-behavior sequence modeling module is implemented as follows: the initial representation E(v) after the neighbor item sequence is embedded is E(v1),...,E(v2). n )} as input, the dependencies between multiple behaviors are captured in the neighbor interaction sequence through multi-behavior multi-head self-attention, and the behavior order information is injected into the neighbor interaction sequence through multi-behavior relative position encoding to explore the functional relationship between behaviors at different positions. Finally, the neighbor sequence representation E containing heterogeneous behaviors and order information is obtained. l (v) = {E l (v1),...,E l (v n )}; The time-aware graph aggregation module first uses time-aware behavior encoding to assign time and behavior information to the sequence representation of neighboring nodes, namely, the neighbor sequence representation E l (v) Then get the node representation z of injected timing and behavior information i (t); Then, the time span and behavior-aware sequence node representations are aggregated through the time graph attention mechanism to obtain the final target node representation 2. A personalized recommendation method based on multi-behavior association modeling and interest boundary perception as claimed in claim 1, characterized in that: The specific method of capturing the dependency between multiple behaviors in the neighbor interaction sequence through multi-behavior multi-head self-attention is as follows: the initial representation E(v) of the input neighbor item sequence is first subjected to a behavior-specific linear mapping to perform q, k, v decomposition: {q,k,v}={f q ,f k ,f v }(E(v),b) where f q ,f k ,f v It is a linear transformation method that linearly maps the original input tensor dimension to the model multi-head dimension through a linear transformation matrix; b is the one-hot encoding of the behavior sequence; Then, when calculating the original attention score, the behavior attention weight matrix is ​​introduced to capture the different effects of other position behaviors on the current behavior, and the relative position encoding is introduced to add order information to the sequence: Where i, j represent the position in the sequence, q[i], k[j] are the decomposed q, k tensors at the corresponding positions, The different parameter matrices constructed for each behavior pair capture the multi-behavior dependencies between different positions. d is the scaling factor in the dot product attention mechanism to prevent the dot product value from being too large, which is the same as the dimension of the query vector q. P[i,j] is the multi-behavior position encoding, which injects behavior order information into the sequence. The final raw attention score A[i,j] between positions i and j is obtained after adding and passing through Softmax. Finally, the final attention score is calculated by multi-head concatenation, and then passes through the MLP with specific behavior, layer normalization and residual connection: S l [i]=Concat(S1[i],...,S h [i]) E l (v i )=MLPs(S l [i],b) S l [i]=LayerNorm(E(v)+S l (i)) E l (v)=LayerNorm(S l (i)+E l (v)) v j is the value vector of the heterogeneity of behaviors at different positions, S[i] is the attention output, For each behavior pair, The parameter matrices of the same dimension capture the heterogeneous relationships between the behaviors when aggregated; Concat splicing operation, S l [i] is the multi-head representation obtained after the splicing operation; E l (v) is the final neighbor sequence representation containing behavioral heterogeneity and sequential information; LayerNorm is a regularization method.

3. A personalized recommendation method based on multi-behavior association modeling and interest boundary perception as claimed in claim 2, characterized in that: The specific implementation method of the multi-behavior relative position coding to inject behavior order information into the neighbor interaction sequence to explore the functional relationship between behaviors at different positions is: Each encoding function Corresponding to a behavior type pair, it encodes the order relationship between the two behavior types; is the behavior type pair (b i ,b j ) The specific meaning of the encoding function is expressed as follows: when the relative distance is close, an independent position is assigned to each position. As the relative distance increases, multiple positions share one position until the relative distance reaches a threshold and is truncated. The specific segmentation processing formula is expressed as follows: Where i and j are the indices of a pair of behaviors in the behavior sequence, ji is the relative distance between j and i, and n b is the number of position codes after segmentation, that is, the total number of independent positions and shared positions, and n is the maximum relative distance, that is, the length of the sequence.

4. The personalized recommendation method based on multi-behavior association modeling and interest boundary perception as claimed in claim 3, characterized in that: The time-aware behavior encoding endows the sequence representation with time and behavior information, and introduces a continuous time mapping function for the time information: where φ(t) is the temporal encoding function, d is the dimension of the temporal encoding, ω1,...,ω d It is an independent and identically distributed random variable from the probability density function distribution P(ω), and t is a specific timestamp; As the time representation of the node, in the multiplication of the self-attention mechanism, the above time function satisfies: φ(t1) T φ(t2)≈φ(t1-t2)=K(t1,t2) Where φ(t1) T refers to the transpose of the time function at time t1, φ(t2) refers to the time function at time t2, φ(t1) T Multiply it with φ(t2) in the self-attention mechanism; K( ) is a defined time kernel function; at the same time, in order to add the behavior information of the interaction edge to the neighbor node representation, the behavior is mapped and spliced ​​into the node representation. Finally, the node after time-aware behavior encoding can be expressed as: z i (t)=[E l (v i )||x 0,i (t i )||φ(t-t i )] Where E l (v i ) is the node v i The node representation obtained after passing through l layers of multi-behavior transformers, x 0,i (t i ) is the target node 0 and node i at t i The interactive behavior type generated at each moment is mapped and embedded, φ(tt i ) contains the time span information of the interaction between nodes 0 and i; z i (t) is the node v i The representation after fusing behavior and time information at time t.

5. The personalized recommendation method based on multi-behavior association modeling and interest boundary perception as claimed in claim 4, characterized in that: The time-series graph attention layer completes the representation aggregation operation of neighbor nodes, and uses the self-attention mechanism to represent the sequence nodes of the previously obtained time-series and behavior perception: Decompose, Z(t) or E l (v1)||x 0,1 (t1)||φ(0),...,E l (v n )||x 0,n (t n )||φ(tt n )] represent the neighbor nodes v1,...,v n The initial representation of the neighbor node representation after fusing the behavior representation and time information, x 0,n (t n ) refers to n The mapping representation of the behavior type generated between node 0 and node n at time φ(tt n ) is the time encoding function about time t: q(t)=[z0(t)]W Q , K(t)=[Z(t)] 1:n W K , V(t)=[Z(t)] 1:n W V Where z0(t) is the representation of the central node 0 after integrating the behavior and time information, [Z(t)] 1:n are neighbor nodes v1,...,v n The representation after integrating behavior and time information, W Q ,W K ,W V It corresponds to the linear transformation matrix, through W Q ,W K ,W V The linear transformation matrix of is multiplied to obtain the decomposed and time-aware query vector q(t), key vector K(t) and value vector V(t), and the final attention score is calculated: q(t) is the time-aware query vector, K(t) T is the transpose of the time-aware key vector, d has the same dimension as the query vector q(t), V(t) is the time-aware value vector, and h0(t) is the final time-aware representation of the central node 0 after aggregating its neighbor nodes; Finally, it is extended to a multi-head self-attention mechanism, through residual connections and layer normalization, and the final target node embedding is expressed as: in are the time-aware representations of node 0 after aggregation on h heads, || represents concatenation, and z0 is the initial representation of node 0 itself; FFN ( ) is a multi-layer neural network for feature transformation, LN refers to the LayerNorm regularization method, It is the final representation of time perception after the time-aware target node 0 aggregates all neighbor nodes before the current moment, and (L+1) means that the current aggregation occurs in the (L+1) hop of the graph.

6. A personalized recommendation method based on multi-behavior association modeling and interest boundary perception as claimed in claim 5, characterized in that: The user interest boundary prediction module introduces an auxiliary score b for each user's interaction behavior at timestamp t u,k (t), which represents the user interest boundary under different behaviors: Where u(t) is the learned user representation at time t, is the representation of the learned behavior edge at time t, It is the multi-behavior relationship matrix learned earlier.

7. The personalized recommendation method of multi-behavior association modeling and interest boundary perception as claimed in claim 6, characterized in that: The loss function of the training process is in point-by-point form, and the scores between user u and the positive term p and the negative term n are expressed as: in represents the representation of user node u after (L+1) hop aggregation at the current time t, represents the representation of the positive item node p after (L+1) hop aggregation at the current time t, It represents the representation of the negative item node n after (L+1) hop aggregation at the current time t. FFN is a multi-layer neural network for dimension conversion. s(u,p) refers to the score between the user and the positive item, and s(u,n) refers to the score between the user and the negative item. The corresponding loss function is expressed as: Where σ is the sigmoid() function, and the loss function penalizes negative samples with scores above the boundary and positive samples with scores below the boundary, b u,k (t) An interest boundary is established for each user behavior interaction, providing different boundaries for different user behaviors; the optimal score distribution is found by adjusting the hyperparameter α to balance the score distribution.

Images