Transform-based personalized sequence recommendation method and system oriented to multiple user behaviors

Abstract

The invention discloses a personalized sequence recommendation method and a personalized sequence recommendation system oriented to multiple behaviors of a user and based on Transform. Firstly, a multi-behavior data fusion method is adopted, various behavior data of a user are comprehensively considered, user interests and preferences can be more comprehensively captured, and the individuation degree of recommendation is improved. Secondly, the time sequence of a user behavior sequence is fully considered, modeling and prediction are conducted on user behaviors through time information, the behavior evolution law and dynamic changes of the user can be well mined, and the timeliness and accuracy of recommendation are improved. By comprehensively considering multi-source information, time sequence modeling and a deep learning method, interests and demands of the user can be more accurately captured, recommendation accuracy and individuation degree are improved, and the user can obtain a recommendation result better meeting actual demands of the user. According to the method, calculation and recommendation can be performed more efficiently when large-scale data and complex conditions are processed, the efficiency and performance of a recommendation system are improved, and rapid and accurate recommendation services are provided for users.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a Transformer-based personalized sequence recommendation method and system for user multi-behavior. Background Art

[0002] In the current shopping context, traditional recommendation systems are usually based on the static assumption that users' shopping behavior patterns are fixed. These systems recommend products to users by capturing their shopping history and preferences. However, in real situations, users' shopping interests and shopping behavior patterns are usually dynamic. For example, users may change their shopping preferences with the change of seasons, holidays, or personal life stages.

[0003] The existing personalized sequence recommendation methods have the following main shortcomings:

[0004] Problem 1. Most of the current research on sequence recommendation suffers from low recommendation accuracy. (1) Most of the current research on sequence recommendation focuses on a single type of shopping behavior (such as click or purchase) without considering multiple types of shopping behaviors. (2) In addition, some multi-behavior sequence recommendation models have proposed methods to study dynamic preferences, but when dealing with shorter user shopping behavior sequences, the recommendation system cannot fully capture the user's true interests, resulting in the final recommendation results not meeting user expectations, which reduces the accuracy of the recommendation.

[0005] Question 2. Some shopping behaviors in the shopping behavior data are considered noise data in the recommendation system. For example, if a user mistakenly jumps to another product page for browsing on a product page, even if he returns to the previous product in a very short time, the wrong shopping behavior will be treated as normal shopping behavior data and included in the data modeling, resulting in reduced recommendation accuracy.

[0006] Question 3. Most of the current research on sequence recommendation only emphasizes the accuracy of the recommendation system, ignoring the long-tail problem caused by the diversity of the recommendation system. Summary of the invention

[0007] In response to the above problems, the present invention proposes a personalized sequence recommendation method and system based on Transformer for multiple user behaviors. For problem 1, the present invention fuses multiple types of shopping behavior data with shopping behavior time information to provide richer input information. And Transformer is used to model the user's shopping behavior by using multiple scales of time to improve the accuracy of personalized recommendations. For problem 2, the present invention improves the accuracy of sequence recommendations by denoising the noise behavior in the data. For problem 3, the present invention uses heterogeneous graphs to model multiple types of shopping behavior data, and uses graph neural networks combined with contrastive learning to solve the long-tail problem, thereby achieving personalized recommendations for products.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] On one hand, the present invention proposes a personalized sequence recommendation method based on Transformer for multiple user behaviors, comprising:

[0010] Step 1: Collect historical shopping behavior data of users, including behavior information in different fields or time periods, and preprocess the data, wherein the preprocessing includes removing noise, and constructing a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence consists of multiple triples, each of which consists of a product, the type of interaction behavior between the user and the product, and the time when the interaction behavior occurs;

[0011] Step 2: Based on the multi-behavior interaction sequence, the multi-behavior context information and time information are integrated with the features of the product representation, and a multi-scale time interval encoder based on the Transformer model is designed to capture the behavior dynamics in the user's multi-behavior interaction sequence, thereby obtaining the dynamic behavior mode of the multi-scale time interval, and integrating the dynamic behavior mode of the multi-scale time interval into the same latent representation space to obtain the final behavior node embedding representation;

[0012] Step 3: From the user's perspective, the connection between products is mined to construct a product-product graph, and a product-behavior heterogeneous graph is generated based on the relationship between products and interactive behaviors. The product-product graph and the product-behavior heterogeneous graph are combined to obtain a combined heterogeneous graph, and data enhancement is performed on the combined heterogeneous graph to obtain an enhanced node view, and the enhanced node view is input into a multi-behavior graph neural network to obtain a product node embedding representation; the multi-behavior graph neural network includes a product and behavior node embedding module, a node aggregation module, and a multi-head attention mechanism module; and the obtained product node embedding representation is further enhanced through graph comparison learning to obtain the final product node embedding representation;

[0013] Step 4: Fuse the final behavior node embedding representation and product node embedding representation to obtain the fused product feature representation;

[0014] Step 5: Calculate the i-th item in the fused item feature representation as item v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

[0015] Furthermore, in step 2, the multi-behavior context information and time information are feature fused with the product embedding representation in the following manner:

[0016]

[0017] where h j is the fused user behavior embedding representation, v j YesS i The embedding representation of the jth item in b j is the shopping behavior type embedding corresponding to the interaction between users and products; j is the embedding time of the interaction behavior; Δt j is the time interval embedding; h j ∈H, Is user w i The corresponding behavior perception interaction sequence S i , J is the total number of products, and d is the dimension of time embedding.

[0018] Furthermore, in step 2, the time interval embedding is obtained in the following manner:

[0019] Δt j =t j -t j-1 ,

[0020]

[0021] where t j is the time when the jth behavior occurs, i.e., the timestamp, and M(·) is the time interval Δt j The mapping is converted into a function of the basic timestamp form, k is the index of the current position, d is the dimension of time embedding, is the intermediate conversion function, and the final time interval embedding is expressed as in is a linear projection matrix.

[0022] Furthermore, the dynamic behavior of the multi-scale time interval is expressed as follows:

[0023]

[0024] in and represents the dynamic behavior of multi-scale time intervals, m represents the total number of time intervals, and n represents the length of the behavior subsequence corresponding to each time interval scale. represents the dynamic behavior-aware aggregator used to capture each time interval scale, is the embedding representation of the user behavior corresponding to the first time interval.

[0025] Furthermore, in step 2, the final behavior node embedding is expressed as follows:

[0026]

[0027] in is the final behavior node embedding representation, Concat represents the concatenation operation, Matrices corresponding to the query, key, and value of the input features, W O is the output transformation matrix; head l represents the lth attention head, there are L of them; It is an input projection function, which is used to transform the input dimension; is the self-attention representation corresponding to H; are the behavior embedding representations corresponding to three different time interval scales, and n1, n2, and n3 respectively represent the lengths of the behavior subsequences corresponding to the three different time interval scales.

[0028] Furthermore, in the commodity and behavior node embedding module, the commodity node is embedded in v in the following manner: i and behavior node embedding b i Perform a linear transformation:

[0029] v′ i =W1v i ,

[0030] b′ i =W2b i ,

[0031] where v′ i and b′ i They represent the product node embedding and behavior node embedding after linear transformation respectively, and W1 and W2 are two different weight matrices.

[0032] Furthermore, in the node aggregation module, processing is performed in the following manner:

[0033]

[0034] in Represents the product node v i The new expression of is the commodity node v i With product node v j The edges between represent is the commodity node v i With behavior node b i The edge between A t1 , A t2 is the attention parameter, LeakyReLU and ReLU are activation functions, is the normalized product-product edge attention coefficient, is the normalized product-behavior edge attention coefficient, Represents the product node v i The set of neighboring commodity nodes, Represents the product node v i The set of neighbor behavior nodes.

[0035] Furthermore, in step 4, the final behavior node embedding representation and the product node embedding representation are fused in the following manner:

[0036] A1=Att(h i )=softmax(W a ·h i α T )

[0037] A2=Att(v i )=softmax(W a ·v i α T )

[0038]

[0039] Where W a and α T is a shared parameter that can be trained; h i ,v i Respectively represent the behavior feature representation and product feature representation at the i-th position; A1 and A2 represent h i and v i The attention coefficient r i Represents the fused product feature representation at the i-th position.

[0040] Furthermore, in step 5, the ith product in the fused product feature representation is calculated as follows: j Probability:

[0041]

[0042] where y i,j Yes i and all possible product feature representations v j The inner product between .

[0043] On the other hand, the present invention proposes a Transformer-based personalized sequence recommendation system for multiple user behaviors, comprising:

[0044] A data collection and preprocessing unit, used to collect historical shopping behavior data of users, including behavior information in different fields or time periods, and preprocess the data, wherein the preprocessing includes removing noise, and constructing a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence is composed of multiple triples, each of which is composed of a product, the type of interaction behavior between the user and the product, and the time when the interaction behavior occurs;

[0045] A dynamic behavior modeling unit for multi-scale time intervals is used to fuse multi-behavior context information and time information with product representation features based on the multi-behavior interaction sequence, design a multi-scale time interval encoder based on the Transformer model to capture the behavior dynamics in the user multi-behavior interaction sequence, and then obtain the dynamic behavior of multi-scale time intervals, and integrate the dynamic behavior of multi-scale time intervals into the same latent representation space to obtain the final behavior node embedding representation;

[0046] A global multi-behavior heterogeneous graph comparison learning unit is used to mine the connections between commodities from the user's perspective to construct a commodity-commodity graph, generate a commodity-behavior heterogeneous graph based on the relationship between commodities and interactive behaviors, combine the commodity-commodity graph and the commodity-behavior heterogeneous graph to obtain a combined heterogeneous graph, perform data enhancement on the combined heterogeneous graph to obtain an enhanced node view, and input the enhanced node view into a multi-behavior graph neural network to obtain a commodity node embedding representation; the multi-behavior graph neural network includes a commodity and behavior node embedding module, a node aggregation module, and a multi-head attention mechanism module; and further enhance the obtained commodity node embedding representation through graph comparison learning to obtain a final commodity node embedding representation;

[0047] The self-attention view fusion unit is used to fuse the final behavior node embedding representation and the product node embedding representation to obtain the fused product feature representation;

[0048] The prediction recommendation unit is used to calculate whether the i-th product in the fused product feature representation is product v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention first adopts a multi-behavior data fusion method, comprehensively considering the user's clicks, purchases, favorites and other behavioral data, which can more comprehensively capture the user's interests and preferences and improve the personalization of recommendations. Secondly, the temporal nature of the user behavior sequence is fully considered, and the user behavior is modeled and predicted by encoding the time information through Transformer, which can better explore the user's behavior evolution law and dynamic changes, and improve the timeliness and accuracy of recommendations. By comprehensively considering multi-source information, temporal modeling and deep learning methods, the present invention can more accurately capture the user's interests and needs, improve the accuracy and personalization of recommendations, and enable users to obtain recommendation results that better meet their actual needs. The present invention can more efficiently perform calculations and recommendations when processing large-scale data and complex situations, improve the efficiency and performance of the recommendation system, and provide users with fast and accurate recommendation services. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a Transformer-based personalized sequence recommendation method for multiple user behaviors according to an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a multi-scale time interval encoder based on a Transformer model provided in an embodiment of the present invention;

[0053] Figure 3 A flowchart of constructing a product-product map provided by an embodiment of the present invention;

[0054] Figure 4 A commodity-behavior heterogeneous graph provided for an embodiment of the present invention;

[0055] Figure 5 A new heterogeneous graph generated by combining a product-product graph provided in an embodiment of the present invention;

[0056] Figure 6 The figure is a schematic diagram of the architecture of a Transformer-based personalized sequence recommendation system for multiple user behaviors according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0058] like Figure 1 As shown in FIG. 1 , a personalized sequence recommendation method based on Transformer for multiple user behaviors includes:

[0059] S101: Collect historical shopping behavior data of users, including behavior information in different fields or time periods, and preprocess the data, wherein the preprocessing includes removing noise, and constructing a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence is composed of multiple triples, each of which is composed of a product, an interaction behavior type between the user and the product, and the time when the interaction behavior occurs;

[0060] S102: Based on the multi-behavior interaction sequence, the multi-behavior context information and time information are integrated with the features of the product representation, and a multi-scale time interval encoder based on the Transformer model is designed to capture the behavior dynamics in the user multi-behavior interaction sequence, thereby obtaining the dynamic behavior mode of the multi-scale time interval, and integrating the dynamic behavior mode of the multi-scale time interval into the same latent representation space to obtain the final behavior node embedding representation;

[0061] S103: From the user's perspective, the connection between products is mined to construct a product-product graph, a product-behavior heterogeneous graph is generated based on the relationship between products and interactive behaviors, the product-product graph and the product-behavior heterogeneous graph are combined to obtain a combined heterogeneous graph, data enhancement is performed on the combined heterogeneous graph to obtain an enhanced node view, and the enhanced node view is input into a multi-behavior graph neural network to obtain a product node embedding representation; the multi-behavior graph neural network includes a product and behavior node embedding module, a node aggregation module and a multi-head attention mechanism module; and the obtained product node embedding representation is further enhanced through graph comparison learning to obtain a final product node embedding representation;

[0062] S104: Fusing the final behavior node embedding representation and the product node embedding representation to obtain a fused product feature representation;

[0063] S105: Calculate the ith product in the fused product feature representation to be product v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

[0064] The method specifically includes:

[0065] 1. Data collection and preprocessing:

[0066] Collect the user's historical shopping behavior data, including behavior information in different fields or time periods. Then preprocess the data, such as removing noise, processing missing values, and building a user-item association matrix. The specific operation of removing noise is as follows: set a timestamp, judge the time of repeated behavior, and regard repeated behaviors between two timestamps as invalid behaviors.

[0067] Multi-behavior interaction sequence: In a sequential recommendation scenario, there is a group of users u i ∈U, where |U|=I; commodity v j ∈V, where |V| = J. For a single user u i , the present invention defines a multi-behavior interaction sequence S i =[(v i,1 , t i,1 , b i,1 ),…,(v i,m , t i,m , b i,m ),…,(v i,M , t i,M , b i,M )], where t i,m Indicates the time when the interaction occurs. The present invention defines b i,m To express u i and S i The mth item in the i,m ) (e.g., page browsing, adding to favorites, adding to shopping carts, and purchasing behaviors in e-commerce platforms). M is a multi-behavior interaction sequence S obtained by sorting the behavior interactions in time. i Length: |S i |=M.

[0068] In the sequential recommendation scenario, the predicted user behavior is defined as the target behavior (for example, the user's purchase behavior in an e-commerce platform); other types of user behaviors are used as auxiliary behaviors to learn the user's preference for the target behavior. i The multi-behavior interaction sequence S i The above content is formalized into:

[0069] Input: Each user u i ∈U multi-behavior interaction sequence S i =[(v i,1 , t i,1 , b i,1 ),…,(v i,m , t i,m , b i,m ),…,(v i,M , t i,M , b i,M )].

[0070] Output: A product v that is used to predict the M+1th position that each user will purchase in the future M+1 The learning function of .

[0071] 2 Modeling of dynamic behavior of multi-scale time intervals:

[0072] (1) Shopping behavior-aware contextual embedding layer:

[0073] In order to further capture the changes in user dynamic behaviors, the time interval between behaviors is added on the basis of injecting multi-behavior context information and behavior time information into the product representation. Single product information and corresponding shopping behavior context and behavior time context are jointly encoded. And the embedded representation is further enhanced by using the time enhancement encoder. According to the position encoding operation in the Transformer, the time interval is mapped into the basic time representation:

[0074]

[0075] where t i is the timestamp of the ith behavior (i.e., the time of occurrence), and M(·) is the generation time interval Δt i Some form of function (e.g., hours, minutes, seconds, etc.), k is the index of the current position, d is the dimension of time embedding, in order to keep all the results after encoding within a reasonable range to maintain the stability of the model, divided by Perform normalization. Δti is the intermediate conversion function. Finally, through a linear projection matrix W t To learn: in Finally, we combine the multi-behavior context information and time information with the product embedding representation v j The features of j :

[0076]

[0077] where v j YesS i The embedding representation of the jth item in b j is the shopping behavior type embedding corresponding to the interaction between users and products; j is the embedding time of the interaction behavior, t j According to the above Δt i Encoding is done in the following way. j ∈H, Is user u i The corresponding behavior perception interaction sequence S i The representation matrix of .

[0078] (2) Multi-scale behavior temporal Transformer layer:

[0079] In the shopping recommendation scenario, user-product shopping preferences may have a variety of shopping behaviors over time. For example, users may buy different categories of products (such as clothing, food, daily necessities, and electronic products) according to their needs in different time periods. Some of these products are necessities for users' daily lives. After they are consumed, they will be purchased immediately. The consumption of products will have a periodic pattern (long-term interest); some products are suddenly of interest to users in a certain period of time (short-term interest). In order to further understand the correlation between behaviors, for example, if a user confirms the purchase after only viewing the behavior in a short period of time, it means that the user currently likes the product very much, otherwise it means that the user's preference for the product is relatively weak. We consider the time interval of user interaction behavior to capture the different preferences of users for products. In order to deal with the above problems, we use the multi-scale of behavior time intervals to describe the user's behavior pattern on the basis of capturing long-term and short-term behavior. By designing a multi-scale time interval encoder based on the Transformer model, we can capture the user's multi-behavior interaction sequence S i (u i ∈U) to more comprehensively capture the changes in user interests, such as Figure 2 shown.

[0080] In the Transformer framework, self-attention is a key component for performing correlation information aggregation between focused data points, which is used to capture the complex dependencies between items in the sequence. The standard self-attention mechanism has some challenges in dealing with the complexity and memory of computing long sequences and short sequences, so the self-attention representation of the low-rank self-attention mechanism (where no secondary attention operation is required) is as follows:

[0081]

[0082] in is a learnable projection embedding transformation matrix, and M and N are used for low-rank embedding transformation.

[0083] In order to enable the model to effectively learn multi-scale time intervals, a hierarchical structure-enhanced Transformer is used to capture the behavioral dynamics of specific time interval scales. The embedding representation vector of is, m represents the total number of time intervals, is the embedding representation of the user behavior corresponding to the first time interval.

[0084] Define a granularity-aware aggregator of intervals to generate a specific time interval representation that reflects the dynamic behavior during that period of time. Each time interval scale corresponds to a behavior subsequence representation as follows:

[0085]

[0086] in and represents the dynamic behavior of multi-scale time intervals, n represents the length of the behavior subsequence corresponding to each time interval scale, represents a dynamic behavior-aware aggregator for capturing each time interval scale. Aggregation of embeddings is performed using average pooling. The behavior-aware feature representation of the time interval is then fed into a self-attention layer to represent the behavior of each time interval scale as:

[0087]

[0088] The layered Transformer we use is designed with three different time interval scales. Therefore, the multi-scale Transformer layer can generate four multi-scale sequential behavior embeddings: n1, n2, and n3 represent the lengths of the behavior subsequences corresponding to three different time interval scales.

[0089] (3) Multi-scale behavior temporal fusion layer:

[0090] In order to integrate the dynamic behaviors of multi-scale time intervals into the same latent representation space, an input projection function is used Will The dimension is converted to Dimension. The above behaviors are embedded and aggregated:

[0091]

[0092] Where ∥ represents the concatenation operation on different embedding vectors.

[0093] In order to improve the learning of richer feature representations in multidimensional space for complex dynamic behavior interaction sequences, H is projected into the latent representation space of l and head-specific attention operations are performed in parallel:

[0094]

[0095] in is the final behavior node embedding representation, Concat represents the concatenation operation, Matrices corresponding to the query, key, and value of the input features, W O is the output transformation matrix, head l Represents the lth attention head, of which there are L.

[0096] (4) Nonlinear functions in feedforward neural networks:

[0097] A feed-forward neural network is used in the multi-scale Transformer to add nonlinearity to the newly generated representation. The nonlinear transformation layer represents:

[0098]

[0099] In the feedforward neural network layer, two layers of nonlinear transformation are used, and Swish(·) is integrated as the intermediate nonlinear activation function. are the learning parameters of the weight matrix and bias vector. Where i represents the i-th multi-scale Transformer layer.

[0100] 3 Global Multi-Behavior Heterogeneous Graph Comparative Learning:

[0101] While solving the long-tail problem in sequential recommendation through this module, we also need to consider (1) when considering multi-order behavioral dependencies, we need to better understand the comprehensive relationship between different types of shopping behaviors, rather than just a simple one-to-one association. For example, when we decide to recommend specific products to users to meet their potential purchase preferences, it will be very helpful to study the user's past shopping behavior interactions with the product (such as browsing pages, adding to favorites). Because users are more likely to add products they are interested in to their favorites before finally purchasing, we can better understand their purchase intentions by analyzing this shopping behavior. This analysis of multi-order behavioral dependencies can provide a more comprehensive understanding of user shopping behavior patterns, thereby improving the effectiveness of personalized recommendation systems. (2) In personalized multi-behavior interaction patterns, different users may exhibit different shopping behavior patterns and show different correlations in different types of user-product shopping behavior interactions. In e-commerce systems, some users tend to add a large number of products to their favorite lists or shopping carts, but only a few of them will eventually be purchased. In contrast, other users will only mark the products of interest as favorites when they really intend to buy them. Therefore, this complex and personalized multi-behavior pattern requires our model to capture the dependencies between different types of shopping behaviors. For personalized recommendation systems, it is crucial to understand and analyze such diverse user behavior patterns in order to better predict user needs and interests and provide recommendation results that are more in line with user personalized needs.

[0102] In order to solve the above problems, the present invention proposes to build global multi-behavior on heterogeneous graph neural structure. Heterogeneous graph can capture the complex relationship between different types of nodes. In multi-behavior sequence recommendation, the relationship between users and items is not limited to one shopping behavior, but may include multiple interactions such as purchase, browsing, and collection. Heterogeneous graph can better model these relationships and provide more accurate recommendation inspiration. The present invention uses heterogeneous graph structure to capture high-order multi-behavior dependencies.

[0103] (1) Multi-behavior data heterogeneous graph construction layer:

[0104] 1. Product-Product Image:

[0105] When different users interact with the same item, different meanings may be expressed. The present invention mines the connections between items from the user's perspective to construct a product-product graph, such as Figure 3 shown.

[0106] 2. Commodity-behavior heterogeneity graph:

[0107] In order to capture the dependencies between multiple shopping behaviors of a single product, a heterogeneous graph structure is further generated based on the observation of multiple types of shopping interactions between products (e.g., page views, adding to carts). like Figure 4 As shown in Figure 2, where the product node v∈V and the shopping behavior node b∈Β.

[0108] 3. Fusion of two images:

[0109] In order to strengthen the embedded representation of products, this module combines the product-product graph and the product-shopping behavior heterogeneous graph, so that the heterogeneous graph has an additional edge relationship between products, such as Figure 5 shown.

[0110] (2) Data enhancement layer of heterogeneous graph structure:

[0111] Three algorithms: random node dropping, random edge removal, and random walks are designed in the graph structure to create different node views.

[0112]

[0113] where two random selections s1 and s2 are independently applied to the graph And create a node and Two related views of .

[0114] 1. Random node drop:

[0115] Randomly select some nodes (product nodes or behavior nodes) in the original graph and discard them. Remove the selected nodes from the graph and delete all edges between these nodes and other nodes. The above operation generates a new subgraph that lacks some nodes and edges compared to the original graph.

[0116] With probability ρ, each node is dropped from the graph along with its connected edges. In particular, s1 and s2 can be modeled as:

[0117]

[0118] in are two masking vectors that are applied to the node set Generate two subgraphs.

[0119] 2. Random edge removal:

[0120] Remove the edge in the graph with probability ρ. The two independent processes are expressed as:

[0121]

[0122] where M1,M2∈{0,1} ε are two masking vectors on the edge set E. Only some of the connections within the neighborhood contribute to the node representation.

[0123] 2. Random Walk:

[0124] Since random node removal and random edge removal generate a shared subgraph for all graph convolutional layers, in order to allow different layers to use different subgraphs, a random walk method can be used to construct a subgraph for each type of node.

[0125]

[0126] in and It is the mask vector of the edge set E on the lth layer, corresponding to different types of edges in the heterogeneous graph.

[0127] (3) Multi-behavior graph neural network layer:

[0128] In the current heterogeneous graph, there is a direct association between product nodes and behavior nodes, and there is also an association between product nodes. In order to consider these different types of edges at the same time, information can be aggregated more comprehensively.

[0129] 1. Product and behavior node embedding module:

[0130] Constructing a heterogeneous graph with behaviors and products as nodes Where V′=V∪B includes the product node set V and the behavior node set B. Each product node is embedded Each behavior node embeds Transform the product node embedding and behavior node embedding respectively.

[0131] Product node linear transformation: v′ i =W1v i ;

[0132] Behavior node linear transformation: b′ i =W2b i ;

[0133] in and are two different weight matrices defined.

[0134] 2. Node aggregation module in graph neural network:

[0135] When calculating the attention coefficient, it is necessary to consider the edges between products and the edges between items and behaviors separately.

[0136] The edge between product nodes (product-product edge) indicates:

[0137]

[0138] Each of the links connects the product node v i and product node v j Edge Use an attention parameter A t1 . LeakyReLU is the activation function.

[0139] The edge between a product node and a behavior node (product-behavior edge) represents:

[0140]

[0141] Each of the links connects the product node v i and behavior node b i Edge Using another attention parameter A t2 .

[0142] Normalization of product-product edge attention coefficient:

[0143]

[0144] Normalization of product-behavior edge attention coefficient:

[0145]

[0146] in Represents the product node v i The set of neighboring commodity nodes, Represents the product node v i The set of neighbor behavior nodes.

[0147] Aggregate neighbor node information:

[0148] Use the normalized attention coefficient to weight the information of neighbor nodes and get the product node v i The new representation, the nonlinear activation function we use is ReLU:

[0149]

[0150] 3. Multi-head attention mechanism module:

[0151] In order to enhance the stability and expressiveness of the model, a multi-head attention mechanism is used. Repeat the above steps N times, each time obtaining a new set of node representations, and finally obtaining K sets of new feature representations, which are then averaged:

[0152]

[0153] in, and and are the attention coefficient of the kth attention head, the feature representation of the product node and the behavior node. Finally, the output of the graph neural network is each product node v i New feature representation

[0154] (4) Use of contrastive learning:

[0155] The original heterogeneous graph As a heterogeneous graph The heterogeneous graph after data enhancement is used as a heterogeneous graph Heterogeneous Graph and heterogeneous graphs The above graph neural model is used to obtain the product feature representations of the two graphs. and The feature representation of the same product in two heterogeneous graphs is used as a positive sample pair The feature representations of different products on two heterogeneous graphs are used as negative sample pairs Among them, product v i The feature representation and product v j The feature representation It is the feature representation of two different products on two heterogeneous graphs. We use the SimCLR framework and the InfoNCE contrast loss to maximize the consistency of positive sample pairs and minimize the consistency of negative sample pairs.

[0156] Loss function formula:

[0157]

[0158] Where sim(·) represents the similarity between two feature representations, using the cosine similarity function:

[0159]

[0160] τ is a hyperparameter used to control the scale of similarity measurement. The best feature representation vector is obtained by comparing the loss function

[0161] 4 Self-Attention View Fusion Layer:

[0162] In order to learn h i and v i The importance of feature representations can be adjusted dynamically to fuse these feature representations, and an attention mechanism is used to fuse the feature representations of different views.

[0163]

[0164] Where W a and α T is a shared parameter that can be trained; h i ,v i Respectively represent the behavior feature representation and product feature representation at the i-th position; A1 and A2 represent h i and v i The attention coefficient r i Represents the fused product feature representation at the i-th position.

[0165] 5 prediction recommendation layers:

[0166] The ith product in the fused product feature representation is product v j Probability:

[0167]

[0168] where v j Is a commodity j The feature representation of y i,j Yes i and all possible product feature representations v j Finally, we calculate the probability y i,j Sort all possible products and select the product with the highest probability as the recommendation result.

[0169] To optimize the accuracy of model predictions, the cross entropy loss and the Adam optimizer are used together:

[0170]

[0171] Based on the above embodiments, Figure 6 As shown, the present invention also proposes a Transformer-based personalized sequence recommendation system for multiple user behaviors, including:

[0172] A data collection and preprocessing unit is used to collect historical shopping behavior data of users, including behavior information in different fields or time periods, preprocess the data, and construct a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence is composed of multiple triples, each of which is composed of a product, the type of interaction behavior between the user and the product, and the time when the interaction behavior occurs;

[0173] A dynamic behavior modeling unit for multi-scale time intervals is used to fuse multi-behavior context information and time information with product representation features based on the multi-behavior interaction sequence, design a multi-scale time interval encoder based on the Transformer model to capture the behavior dynamics in the user multi-behavior interaction sequence, and then obtain the dynamic behavior of multi-scale time intervals, and integrate the dynamic behavior of multi-scale time intervals into the same latent representation space to obtain the final behavior node embedding representation;

[0174] A global multi-behavior heterogeneous graph comparison learning unit is used to mine the connections between commodities from the user's perspective to construct a commodity-commodity graph, generate a commodity-behavior heterogeneous graph based on the relationship between commodities and interactive behaviors, combine the commodity-commodity graph and the commodity-behavior heterogeneous graph to obtain a combined heterogeneous graph, perform data enhancement on the combined heterogeneous graph to obtain an enhanced node view, and input the enhanced node view into a multi-behavior graph neural network to obtain a commodity node embedding representation; the multi-behavior graph neural network includes a commodity and behavior node embedding module, a node aggregation module, and a multi-head attention mechanism module; and further enhance the obtained commodity node embedding representation through graph comparison learning to obtain a final commodity node embedding representation;

[0175] The self-attention view fusion unit is used to fuse the final behavior node embedding representation and the product node embedding representation to obtain the fused product feature representation;

[0176] The prediction recommendation unit is used to calculate whether the i-th product in the fused product feature representation is product v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

[0177] In summary:

[0178] 1) Learning users’ dynamic preferences is a very important part of the research on sequential recommendations. Dynamic preferences are captured by users’ interests that change over time. (1) Most existing research on sequential recommendations focuses on shopping behaviors of a single type of shopping interaction (e.g., “click” or “buy”), without considering multiple types of user-item relationships. For example, in actual online platforms, user behaviors often show time dependence and multiple types, involving different types of user-item shopping interactions, such as “page browsing”, “add to cart”, “favorite”, and “buy”. (2) Secondly, some multi-behavior sequential recommendation models have proposed methods to study dynamic preferences, but inaccurate recommendations may occur when describing user preferences with shorter behavior sequences. For example, there are fewer items interacting with users or fewer interactive shopping behaviors, resulting in insufficient basis for recommendations (sparse data).

[0179] In order to solve the above-described problems, the present invention combines multi-behavior data with behavioral time context data to provide richer input information to improve the modeling ability and prediction accuracy of the model for multi-behavior sequence data. Considering that there are many types in the data (users, commodities, viewing, purchasing and other interactive behaviors). In addition, in actual recommendation scenarios, user-commodity interaction preferences may present a transition pattern of multi-scale behaviors over time. For example, users often buy different categories of commodities (such as daily necessities, clothing, digital devices), and these commodities will be subject to different user preferences. There will be different shopping interaction behaviors in different time periods. In order to solve the behavioral transition trends corresponding to different time periods, for example, if the user confirms the purchase only after viewing the behavior in a short period of time, it means that the user currently likes the commodity very much, otherwise it means that the user's preference for the commodity is relatively weak. The present invention uses Transformer to model the user's interactive behavior to improve the accuracy of the dynamic preference of personalized sequence recommendation.

[0180] 2) There will be some repeated interactive behaviors (auxiliary behaviors) between certain users and products in the multi-behavior data of sequential recommendation. Based on actual user habits, the present invention finds that the reason for this phenomenon is most likely caused by incorrect operations of users. These behaviors are noise data in the recommendation system, and failure to process them will affect the accuracy of the recommendation system. Previous studies usually have two approaches to repeated auxiliary behaviors. Approach 1: Treat all repeated auxiliary behaviors as valid behaviors. Approach 2: Randomly eliminate the repeated auxiliary behaviors to one. However, the above two approaches are too extreme.

[0181] In order to better identify invalid repeated behaviors, the present invention is based on the habits of users in real life. Incorrect operation behaviors will return to the previous behavior in a very short time. By setting a timestamp to judge the time of repeated behaviors, the behaviors within the time range are regarded as invalid behaviors. In this way, invalid behaviors (noise) are removed to improve the accuracy of sequence recommendations.

[0182] 3) Most of the current studies on sequence recommendation only emphasize the accuracy of the recommendation system, while ignoring the diversity of the recommendation system, which leads to the long tail problem. The present invention uses heterogeneous graphs to model multi-behavior data, and uses graph neural networks and graph contrast learning to solve the above problems.

[0183] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A personalized sequence recommendation method based on Transformer for multiple user behaviors, characterized in that: include: Step 1: Collect historical shopping behavior data of users, including behavior information in different fields or time periods, and preprocess the data, wherein the preprocessing includes removing noise, and constructing a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence consists of multiple triples, each of which consists of a product, the type of interaction behavior between the user and the product, and the time when the interaction behavior occurs; Step 2: Based on the multi-behavior interaction sequence, the multi-behavior context information and time information are integrated with the features of the product representation, and a multi-scale time interval encoder based on the Transformer model is designed to capture the behavior dynamics in the user's multi-behavior interaction sequence, thereby obtaining the dynamic behavior mode of the multi-scale time interval, and integrating the dynamic behavior mode of the multi-scale time interval into the same latent representation space to obtain the final behavior node embedding representation; Step 3: From the user's perspective, we mine the connections between products to construct a product-product graph. Based on the relationship between products and interactive behaviors, we generate a product-behavior heterogeneous graph. We combine the product-product graph and the product-behavior heterogeneous graph to obtain a combined heterogeneous graph. We perform data enhancement on the combined heterogeneous graph to obtain an enhanced node view. We input the enhanced node view into the multi-behavior graph neural network to obtain a product node embedding representation. The multi-behavior graph neural network includes a commodity and behavior node embedding module, a node aggregation module and a multi-head attention mechanism module; and through graph comparison learning, the obtained commodity node embedding representation is further enhanced to obtain the final commodity node embedding representation; Step 4: Fuse the final behavior node embedding representation and product node embedding representation to obtain the fused product feature representation; Step 5: Calculate the i-th item in the fused item feature representation as item v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

2. The Transformer-based personalized sequence recommendation method for user multi-behavior according to claim 1, characterized in that: In step 2, the multi-behavior context information and time information are feature fused with the product embedding representation in the following manner: where h j is the fused user behavior embedding representation, v j YesS i The embedding representation of the jth item in b j is the shopping behavior type embedding corresponding to the interaction between users and products; j It is the embedding of the time when the interactive behavior occurs; Δt j is the time interval embedding; h j ∈H, Is user u i The corresponding behavior perception interaction sequence S i , J is the total number of products, and d is the dimension of time embedding.

3. The Transformer-based personalized sequence recommendation method for user multi-behavior according to claim 2 is characterized in that: In step 2, the time interval embedding is obtained in the following manner: Δt j =t j -t j-1 , where t j is the time when the jth behavior occurs, i.e., the timestamp, and M(·) is the time interval Δt j The mapping is converted into a function of the basic timestamp form, k is the index of the current position, d is the dimension of time embedding, is the intermediate conversion function, and the final time interval embedding is expressed as in is a linear projection matrix.

4. The Transformer-based personalized sequence recommendation method for user multi-behavior according to claim 1, characterized in that: The dynamic behavior of the multi-scale time interval is expressed as follows: in and represents the dynamic behavior of multi-scale time intervals, m represents the total number of time intervals, and n represents the length of the behavior subsequence corresponding to each time interval scale. represents the dynamic behavior-aware aggregator used to capture each time interval scale, is the embedding representation of the user behavior corresponding to the first time interval.

5. The Transformer-based personalized sequence recommendation method for user multi-behavior according to claim 1, characterized in that: In step 2, the final behavior node embedding is expressed as follows: in is the final behavior node embedding representation, Concat represents the concatenation operation, Matrices corresponding to the query, key, and value of the input features, W o is the output transformation matrix; head l represents the lth attention head, there are L of them; It is an input projection function, which is used to transform the input dimension; is the self-attention representation corresponding to H; are the behavior embedding representations corresponding to three different time interval scales, and n1, n2, and n3 respectively represent the lengths of the behavior subsequences corresponding to the three different time interval scales.

6. The method for personalized sequence recommendation based on Transformer for multiple user behaviors according to claim 1, characterized in that: The commodity and behavior nodes are embedded in the module, and the commodity nodes are embedded in v in the following way i and behavior node embedding b i Perform a linear transformation: v′ i =W1v i , with i =W2b i , where v′ i and b′ i They represent the product node embedding and behavior node embedding after linear transformation respectively, and W1 and W2 are two different weight matrices.

7. The method for personalized sequence recommendation based on Transformer for multiple user behaviors according to claim 6, characterized in that: In the node aggregation module, processing is performed in the following manner: in Represents the product node v i The new expression of is the commodity node v i With product node v j The edges between represent is the commodity node v i With behavior node b i The edge between A t1 , A t2 is the attention parameter, LeakyReLU and ReLU are activation functions, is the normalized product-product edge attention coefficient, is the normalized product-behavior edge attention coefficient, Represents the product node v i The set of neighboring commodity nodes, Represents the product node v i The set of neighbor behavior nodes.

8. The method for personalized sequence recommendation based on Transformer for multiple user behaviors according to claim 1, characterized in that: In step 4, the final behavior node embedding representation and the product node embedding representation are fused in the following manner: A1=At(h i )=softmax(W a ·h i ·α T ) A2=Att(v i )=softmax(W a ·v i ·α T ) Where W a and α T is a shared parameter that can be trained; h i ,v i Represent the behavior feature representation and product feature representation at the i-th position respectively; A1 and A2 represent h i and v i The attention coefficient r i Represents the fused product feature representation at the i-th position.

9. The method for personalized sequence recommendation based on Transformer for multiple user behaviors according to claim 8, characterized in that: In step 5, the ith product in the fused product feature representation is calculated as follows: j Probability: where y i,j Yes i and all possible product feature representations v j The inner product between .

10. A personalized sequential recommendation system based on Transformer for multiple user behaviors, characterized by: include: A data collection and preprocessing unit, used to collect historical shopping behavior data of users, including behavior information in different fields or time periods, and preprocess the data, wherein the preprocessing includes removing noise, and constructing a multi-behavior interaction sequence for each user based on the preprocessed data; the multi-behavior interaction sequence is composed of multiple triples, each of which is composed of a product, the type of interaction behavior between the user and the product, and the time when the interaction behavior occurs; A dynamic behavior modeling unit for multi-scale time intervals is used to fuse multi-behavior context information and time information with product representation features based on the multi-behavior interaction sequence, design a multi-scale time interval encoder based on the Transformer model to capture the behavior dynamics in the user multi-behavior interaction sequence, and then obtain the dynamic behavior of multi-scale time intervals, and integrate the dynamic behavior of multi-scale time intervals into the same latent representation space to obtain the final behavior node embedding representation; The global multi-behavior heterogeneous graph comparison learning unit is used to mine the connections between products from the user's perspective to construct a product-product graph, generate a product-behavior heterogeneous graph based on the relationship between products and interactive behaviors, combine the product-product graph and the product-behavior heterogeneous graph to obtain a combined heterogeneous graph, perform data enhancement on the combined heterogeneous graph to obtain an enhanced node view, and input the enhanced node view into the multi-behavior graph neural network to obtain a product node embedding representation; The multi-behavior graph neural network includes a commodity and behavior node embedding module, a node aggregation module and a multi-head attention mechanism module; and through graph comparison learning, the obtained commodity node embedding representation is further enhanced to obtain the final commodity node embedding representation; The self-attention view fusion unit is used to fuse the final behavior node embedding representation and the product node embedding representation to obtain the fused product feature representation; The prediction recommendation unit is used to calculate whether the i-th product in the fused product feature representation is product v j According to the calculated probability, all possible products are sorted and the product with the highest probability is selected as the recommendation result.

Images