Sequential recommendation method based on adaptive global graph modeling and intent contrastive learning

Abstract

The present invention relates to the technical field of artificial intelligence optimized operating systems, and specifically discloses a sequence recommendation method based on adaptive global graph modeling and intent contrast learning. The method comprises: sequence global graph construction, module output fusion, sequence recommendation learning, and recommendation result optimization. First, a user historical behavior sequence is obtained, constructed into an association graph, and improved and optimized to obtain and record it as a user behavior sequence global graph, which is injected into a local module and a global module of a sequence encoder respectively, so that the outputs of the local module and the global module are fused, so that the sequence encoder can comprehensively learn more expressive intentions, thereby performing intent contrast learning, comprehensively obtaining user sequence recommendations, and predicting users, thereby completing sequence recommendation based on adaptive global graph modeling and intent contrast learning, thereby improving the recommendation effect in noisy and data sparse conditions, improving user experience, and enhancing the robustness and generalization ability of the sequence recommendation method in multiple dimensions.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence optimized operating systems, and specifically to a sequence recommendation method based on adaptive global graph modeling and intent comparative learning. Background Art

[0002] With the widespread adoption of online recommendation systems, accurately capturing user intent and providing personalized recommendations has become a key research topic. Traditional recommendation systems rely heavily on historical user behavior data. However, this approach often neglects the deeper meaning of user intent and the high-level relationships between items. Therefore, researching recommendation models that combine global information with personalized features is crucial.

[0003] For example, the invention patent with announcement number CN111967941B announces a method for constructing a sequence recommendation model and a sequence recommendation method. The method for constructing a sequence recommendation model includes: constructing a first sequence recommendation model and a second sequence recommendation model, wherein the first sequence recommendation model is provided with multiple hole convolution residual blocks, and the second sequence recommendation model contains lightweight hole convolution residual blocks corresponding one to one to the first sequence recommendation model; training the first sequence recommendation model to obtain a pre-trained first sequence recommendation model; collaboratively training the second sequence recommendation model and the pre-trained first sequence recommendation model, and enabling the second sequence recommendation model to learn the behavior of the pre-trained first sequence recommendation model; and constructing a sequence recommendation model for predicting recommendation items based on the collaboratively trained second sequence recommendation model.

[0004] For example, the invention patent with announcement number CN113762477B announces a method for constructing a sequence recommendation model and a sequence recommendation method, including: constructing an adaptive adjacency matrix of the input sequence, and constructing the first embedding of the project based on the adaptive adjacency matrix; constructing the second embedding of the project based on the adjacency matrix of the graph neural network; constructing the user's local interest model through the attention mechanism based on the first embedding of the project and the second embedding of the project; constructing the user's global interest model and the embedding of the target sequence, and constructing a sequence recommendation model based on the embedding of the target sequence, the user's local interest model and the global interest model; constructing the loss function of the sequence recommendation model based on gradient descent and Bayesian personalized sorting.

[0005] Combining the above technical solutions, we found that there are sequential recommendation technical solutions, whose analysis dimensions of sequential recommendation are relatively single, which may lead to insufficient generalization ability of recommendation results, thereby affecting the prediction of user behavior and ultimately affecting user usage. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention provides a sequence recommendation method based on adaptive global graph modeling and intent contrast learning, which can effectively solve the problems involved in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a sequence recommendation method based on adaptive global graph modeling and intention comparative learning, including: S1. Obtaining historical interaction data between users and items, and sorting them based on rules to obtain user historical behavior sequences, constructing them into association graphs, improving and optimizing the association graphs, and obtaining and recording them as global graphs of user behavior sequences; S2. Dividing the global graph of user behavior sequences into subgraphs of user behavior sequences, combining them with user embeddings, and injecting them into the local modules of the sequence encoder, injecting the global graph of user behavior sequences into the global module of the sequence encoder, fusing the outputs of the local modules and the global modules to obtain representations of user behavior sequences; S3. Extracting user intent from the user behavior sequence through the sequence encoder, performing comparative learning on the representation of the user behavior sequence, performing comparative learning on the representation of the user behavior sequence, and comprehensively obtaining sequence recommendations for users; S4. Based on the user's sequence recommendation, predicting the user, and optimizing the sequence recommendation according to the prediction results, thereby completing sequence recommendation based on adaptive global graph modeling and intention comparative learning.

[0008] As a further method, the rule-based sorting is performed to obtain a user historical behavior sequence, which is constructed as an association graph. The specific analysis process is: the rule is specifically sorted according to the time sequence of each historical interaction time point; the historical interaction data between the user and the item is specifically the interaction data between the user and the item at each historical interaction time point; the user historical behavior sequence is obtained, specifically sorting the interaction data between the user and the item at each historical interaction time point according to the time sequence of each historical interaction time point, and obtaining and recording it as each user historical behavior subsequence, so that the set of each user historical behavior subsequence is called a user historical behavior sequence; the user historical behavior sequence is divided into a project set and a relationship set between the user and the project; the construction as an association graph is specifically connecting the project set in the user historical behavior sequence and the relationship set between the user and the project in a predefined manner, and the specific expression is: association graph , where V is the set of items, is the set of relationships between users and items, It is represented as the relationship between vertex i and vertex j of the e-th edge in the association graph, .

[0009] As a further method, the association graph is improved and optimized to obtain and record it as a global graph of user behavior sequence. The specific analysis process is as follows: the above-mentioned improvement and optimization of the association graph is specifically to optimize the association graph through a lightweight graph convolutional network as a graph encoder to obtain and record it as a global graph of user behavior sequence; the global graph representation of the user behavior sequence is obtained by embedding the association graphs from each L layer of the lightweight graph convolutional network through the graph encoder, as shown below:

[0010] ;

[0011] Where, It is a global graph representation of user behavior sequence, is the association graph representation of layer 0, Represented as the association graph representation of the rth layer, is the association graph representation of the Lth layer, L is a specific layer representation in the lightweight graph convolutional network, ,r represents any layer in the lightweight graph convolutional network.

[0012] As a further method, the output of the local module and the global module are fused to obtain the representation of the user behavior sequence. The specific analysis process is: obtain the position weight of each position point p in the global graph of the user behavior sequence, denoted as ; The output representation of the fusion layer is constructed as:

[0013] ;

[0014] Where, is the output of the fusion layer, is the importance of the item at position p for predicting user intention, Denoted as the local module output of the sequence encoder, is the global module output of the sequence encoder; the representation of the user behavior sequence is obtained, specifically The last vector in is used as the representation of user sequence, where .

[0015] As a further method, the user intention is compared with the representation of the user behavior sequence. The specific analysis process is: obtain two vectors of the global graph of the user behavior sequence in the vector space, which are respectively recorded as and ; Combine the vector space with the vector and The intention prototype with the smallest distance is used as the intention of the two vectors. and The contrastive learning task between the representation of user behavior sequences and user intentions is constructed as follows:

[0016] ;

[0017] Where, It is represented as a contrastive learning between the representation of user behavior sequence and user intention, Represented as a sequence of user behaviors, and Represented as a vector in a vector space, and Expressed as and The intent prototype with the smallest distance is taken as the user intent.

[0018] As a further method, the representation of the user behavior sequence is compared with the representation of other user behavior sequences. The specific analysis process is: obtain the current sequence of the user behavior sequence, recorded as ; For the current sequence , randomly sample from sequences with the same label to obtain sequences with the same intent , using this as a comparison between sequences, the learning task is constructed as follows:

[0019] ;

[0020] Where, For the comparison between user behavior sequences, is the user behavior sequence, and Represented as two vectors in a vector space.

[0021] As a further method, the user-based sequence recommendation is used to predict the user. The specific analysis process is as follows: the user is predicted, specifically given a sequence representation And the item embedding matrix V, the prediction score is calculated as follows:

[0022] ;

[0023] in is the prediction score, , For a given sequence representation, is the transpose of the item embedding matrix, is the determinant representation of the embedding matrix, Expressed as a normalization function; the cross entropy loss is optimized to maximize the probability of correct prediction: ;

[0024] Where, To predict the accuracy of users, is the next real item of interaction of user u, is the predicted item of the next interaction of user u, .

[0025] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0026] (1) The present invention provides a sequence recommendation method based on adaptive global graph modeling and intention contrast learning. First, the user's historical behavior sequence is obtained, constructed into an association graph and improved and optimized, and a global graph of the user behavior sequence is obtained and recorded. The global graph is injected into the local module and the global module of the sequence encoder respectively, so that the outputs of the local module and the global module are fused, so that the sequence encoder can comprehensively learn more expressive intentions, thereby performing intention contrast learning, comprehensively obtaining the user's sequence recommendation, and predicting the user, completing the sequence recommendation of adaptive global graph modeling and intention contrast learning, improving the recommendation effect in noisy and data sparse conditions, improving the user experience, and enhancing the robustness and generalization ability of the sequence recommendation method in multiple dimensions.

[0027] (2) The present invention divides the global graph of user behavior sequences into sub-graphs of each user behavior sequence, combines it with user embedding, and injects it into the local module of the sequence encoder. The outputs of the local module and the global module are fused to obtain a representation of the user behavior sequence. The global information between sequences is effectively utilized rather than just the local information within the sequence, thereby improving the ability to capture the complex intentions of users and enhancing the efficiency of recommendations.

[0028] (3) The present invention extracts user intent from user behavior sequences through a sequence encoder, performs comparative learning on the representation of user behavior sequences, and compares the representation of user behavior sequences with the representation of other user behavior sequences to comprehensively obtain user sequence recommendations. Through sequence-level and intent-level comparative self-supervised learning tasks, user intent is modeled from different dimensions, aiming to reveal semantic patterns in user behavior and improve the accuracy and performance of sequence recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0030] Figure 1 Schematic diagram of the method of the present invention.

[0031] Figure 2 A simplified diagram of the association graph.

[0032] Figure 3 A detailed statistical table of the dataset.

[0033] Figure 4 Performance comparison results comparison chart.

[0034] Figure 5 Comparison chart of visualization results. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] Embodiments of this invention can be applied to platforms including but not limited to e-commerce platforms, video websites, online advertising platforms, and social networking platforms that require rich sequential data generated from historical user data. Through adaptive global graph modeling, it can capture the global relevance between users and items and their underlying intent, thereby enabling recommendations for users' preferences. Intent comparative learning can help platforms understand changes in user interests over time and recommend content that better matches the user's current mood and interests.

[0037] To efficiently capture high-order transition relationships between items and accurately learn user intent, we propose IC4SRec (a sequential recommendation method based on adaptive global graph modeling and contrastive learning of intent), which comprehensively integrates item collaboration information and user intent into the sequential recommendation model from both global and local perspectives. Specifically, this embodiment of the present invention first proposes a global graph information learner that adaptively learns an improved graph from the transition correlations between all items and seamlessly integrates it into the item embeddings. Next, subgraphs are extracted from the global graph corresponding to the user sequence, combined with the user embeddings, and injected into the local module of the sequence encoder in the form of relative position encoding to promote the learning of personalized intent. Furthermore, to avoid over-focusing on recent interactions, this embodiment of the present invention also designs a global module with shared parameters across all sequences. With high-quality item embeddings and an encoder that integrates global and local information, this invention learns accurate user intent from interaction sequences through clustering. Through a sequence-level contrastive self-supervised learning task, we bring sequences with the same intent closer together in the latent space. Simultaneously, this intent-level contrastive self-supervised learning task brings sequences closer to the intent prototype, thereby modeling the most appropriate latent intent for the current user sequence. Finally, the embodiment of the present invention jointly optimizes the entire prediction process in an end-to-end manner through multi-task learning.

[0038] In the embodiment of the present invention, the sequential recommendation method predicts the items that the user will interact with in the next moment based on the user's historical interaction sequence with time sequence information. Assume that the user set and item set are represented as U and V respectively, and each user The interaction sequences of user u are sorted in chronological order, and the historical interaction sequence of user u is expressed as ,in represents the item that user u interacts with for the i-th time, and t is the maximum length of the sequence.

[0039] Reference Figure 1 As shown, the present invention provides a sequence recommendation method based on adaptive global graph modeling and intention contrast learning, including: S1. Obtaining historical interaction data between users and items, and sorting them based on rules to obtain user historical behavior sequences, constructing them into association graphs, improving and optimizing the association graphs, and obtaining and recording them as global graphs of user behavior sequences.

[0040] The historical interaction data between the above-mentioned users and items can be extracted from the user information database of the Internet service provider.

[0041] Specifically, the rule-based sorting is performed to obtain a user historical behavior sequence, which is constructed into a correlation graph. The specific analysis process is as follows:

[0042] The rule is specifically to sort the historical interaction time points in chronological order, wherein the chronological order is the order of the corresponding time points of the interaction behaviors.

[0043] The above historical interaction time points are the historical interaction time points between users and items, which are extracted from the user information database of the Internet service provider.

[0044] The historical interaction data between the user and the item is specifically the interaction data between the user and the item at each historical interaction time point, where the historical interaction data may include playing games at historical interaction time point 1, listening to music at historical interaction time point 2, playing football at historical interaction time point 3, and swimming at historical interaction time point 4.

[0045] The user historical behavior sequence is obtained by sorting the interaction data between the user and the item at each historical interaction time point in chronological order to obtain each user historical behavior subsequence. The collection of each user historical behavior subsequence is called a user historical behavior sequence.

[0046] The user historical behavior sequence is divided into an item set and a relationship set between users and items.

[0047] The above user historical behavior sequence contains a large amount of interaction data between users and items, such as objects. All items that have appeared are extracted from the user historical interaction data, and duplicate items are removed to form an item set. For each record in the user historical interaction data, it is classified into the corresponding user and item according to its user ID and item ID. At the same time, the behavior type and behavior time are recorded to form a complete relationship set.

[0048] The construction is an association graph, specifically dividing the user's historical behavior sequence into a set of items and a set of relationships between users and items, and connecting them in a predefined way. The specific expression is:

[0049] Association diagram , where V is the set of items, is the set of relationships between users and items in all sequences, i.e., the edge set, It is represented as the relationship between vertex i and vertex j of the e-th edge in the association graph, , vertex i can be represented as the starting point of the relationship, and vertex j can be represented as the end point of the relationship.

[0050] The connection is performed in a predefined manner, which may be: : ; : ; : ; : , then the user to The corresponding items v are connected according to their existing relationships, as shown by the arrows, such as Figure 2 As shown, Figure 2 A simplified diagram of the association graph.

[0051] Depend on Figure 2 It can be seen that the association graph is an undirected graph, which means that there may be repeated relationships between users and items in the user's historical behavior sequence, which will cause interference with sequence recommendations when learning user intentions. Therefore, it is necessary to optimize the association graph to effectively reduce the impact of such repeated relationships on learning effects, thereby providing users with more personalized and high-quality recommendation services.

[0052] It should be explained that the above historical interaction data only contains user interaction records. In order to obtain a global perspective that can effectively capture the commonalities and correlations in user behaviors, this embodiment introduces the graph structure learning (GSL) method to learn an adaptive and personalized association graph. Through the GSL method, the association graph obtained in this embodiment can adaptively detect and remove noisy edges, while identifying potential valid edges.

[0053] Taking the user's historical behavior sequence S as an example, for each item in the sliding window Update the edge weights to , where t is set to 2, which is a suitable window size. According to the lightweight graph convolutional network, It can be regarded as a measure of the semantic relevance of the current item to its t-hop neighbors in the sequence. In this embodiment, the above operation is performed on all user sequences, and then the edge weights are normalized as follows:

[0054] ;

[0055] in is the degree of the node in the incidence graph G, Represented as the i-th vertex in the item set V, Represented as the j-th vertex in the item set V, since the association graph can also be regarded as a kind of global collaborative information, and global collaborative information emphasizes the interaction and association between different users and behavioral items, considering the association graph as global collaborative information helps to capture and represent the comprehensive impact of intention in the overall system, which helps to understand the intention more accurately and consider its complex relationship with other sequences. Global collaborative information is usually dynamic and can reflect the changes in the interaction between users and behavioral items over time. In intent modeling, this helps to capture the evolution and changes of user intentions, so as to adapt to different application scenarios more flexibly. In summary, the association graph in this embodiment helps to better carry out subsequent intent modeling.

[0056] Furthermore, the association graph is improved and optimized to obtain and record it as a global graph of user behavior sequences. The specific analysis process is as follows:

[0057] The above-mentioned improvement and optimization of the association graph is to optimize the association graph through a lightweight graph convolutional network as a graph encoder, and obtain and record it as a global graph of user behavior sequence.

[0058] The above optimization can be specifically achieved by using a graph encoder to perform structural learning on the association graph. Structural learning of the association graph can enhance the representation and robustness of the association graph. Therefore, in this embodiment, partial structural perturbation is used to improve the association graph. The adjacency matrix of the association graph can be treated as a free variable, which can be formalized as follows:

[0059] ;

[0060] in , It is a correlation graph. is a learnable perturbation map, is the disturbance intensity, yes OK The matrix representation of the columns, It is the specific value of the item set.

[0061] In a specific embodiment, suppose there is an association graph G, which contains 4 vertices , that is, the item set contains 4 items and 4 edges , that is, there are four types of relationships between users and items. The relationship of the adjacency matrix A of the association graph is shown in Table 1 below and can be expressed as:

[0062] Table 1 Relationship of adjacency matrix A

[0063]

[0064] Then the adjacency matrix A of the association graph is expressed as: In this matrix, A[1][2]=1 means that there is an edge between vertex v1 and vertex v2; A[2][4]=1 means that there is an edge between vertex v2 and vertex v4, and so on.

[0065] At the same time, since the above perturbation graph is a dense graph, in order to reduce the expression In order to reduce the computational complexity of the perturbation graph, this embodiment aims to accelerate the learning of the perturbation graph. Specifically, this embodiment only models a few dominant singular values of the global graph of the user behavior sequence, thereby learning its SVD decomposition, i.e., singular value decomposition, which is an important matrix decomposition method. Specifically, a matrix is decomposed into the product of three matrices: an orthogonal matrix, a diagonal matrix, and the transpose of another orthogonal matrix. The formula is as follows:

[0066]

[0067] in is a learnable perturbation map, is the adjacency matrix of the incidence graph, and are all learnable matrices, is a hyperparameter that determines the rank of the decomposed perturbation graph, Represented as a matrix With the matrix The product of Represented as a matrix With the matrix The product of Represented as a matrix With the matrix After the above operations are completed, this embodiment uses the proven lightweight graph convolutional network as a graph encoder to capture the global collaborative relationship and rewrites its message passing method as follows:

[0068]

[0069] is the association graph representation of the Lth layer, It is the L-1 layer association graph representation.

[0070] The global graph representation of the user behavior sequence is obtained by embedding the weighted sum of the association graphs from each L layer of the lightweight graph convolutional network through the graph encoder, as shown below:

[0071]

[0072] Where, It is a global graph representation of user behavior sequence, is the association graph representation of layer 0, Represented as the association graph representation of the rth layer, is the association graph representation of the Lth layer, L is a specific layer representation in the lightweight graph convolutional network, ,r represents any layer in the lightweight graph convolutional network.

[0073] It should be noted that a single association graph only has explicit item relationships and is affected by noise. In this embodiment, self-supervised contrastive learning is used to model the improved global graph of user behavior sequences to mine implicit relationships between items and generate high-quality item representations.

[0074] Although the graph encoder explicitly models global information through indirect dependencies, directly applying them to the sequence context requires the help of self-supervised learning. Considering that the association graph or the global graph of user behavior sequences should have similar information, this embodiment also incorporates the refined global collaborative information into the item representation through the mutual information maximization criterion (MIM). This can be achieved by modifying the InfoNCE function, which is expressed as follows:

[0075] ;

[0076] in It is represented as the global information encoding loss, S is the user behavior sequence, , , L is a specific level representation in the lightweight graph convolutional network, i is the number of samples in a batch, , B is the number of samples in a batch, τ is the temperature coefficient, and this objective function obtains high-quality item embeddings containing global collaborative information, which helps to better perform intent modeling and sequence modeling in the future.

[0077] The above InfoNCE function is a loss function commonly used in contrastive learning. Its goal is to maximize the similarity between similar samples (positive samples) and minimize the similarity between non-similar samples (negative samples). It enables the model to learn more discriminative feature representations through contrastive learning. Specifically, suppose there is a set of samples represented as , and a positive sample pair ( ) and negative sample pairs ( ),in is with Similar positive samples, is with For dissimilar negative samples, the formula for InfoNCE loss can be expressed as:

[0078] ;

[0079] Among them, sim( )express and The similarity between them is usually calculated using cosine similarity; τ is a temperature parameter used to adjust the scale of similarity in contrastive learning to avoid gradient vanishing or gradient exploding problems.

[0080] S2. Divide the global graph of user behavior sequences into subgraphs of each user behavior sequence, combine them with user embeddings, and inject them into the local module of the sequence encoder. Inject the global graph of user behavior sequences into the global module of the sequence encoder, fuse the outputs of the local module and the global module, and obtain a representation of the user behavior sequence.

[0081] Specifically, the global graph of user behavior sequences is divided into subgraphs of user behavior sequences, combined with user embedding, and injected into the local module of the sequence encoder. The specific analysis process is as follows:

[0082] It should be explained that the purpose of sequential recommendation is to provide personalized suggestions for specific users based on item relationships, and to obtain the user's personalized information based on the global graph of the user's behavior sequence. The user's personalized information refers to a collection of information compiled based on the user's unique identity, historical behavior, interest preferences, social relationships and other data. This information can be used to distinguish the differences between different users and is the basis for realizing personalized information services. Specifically, it can be extracted from the user's browsing history on the website or application.

[0083] The personalized information of the user mentioned above may specifically be user identity characteristics: such as age, gender, occupation, region and other basic information, which can reflect the user's basic attributes and social background.

[0084] Historical behavior data: including users’ browsing history, purchase history, search history, etc. on websites or applications. These data can reflect users’ interests and needs.

[0085] Interest preferences: The user's preference for specific content, products, or services, such as favorite movie genres, music styles, shopping categories, etc.

[0086] Social relationships: users’ friendships and interactions on social media. This information can reflect the user’s social circle and influence.

[0087] In order to make the results of the sequence encoder more consistent with the user's intention, we extract the subgraph of the user sequence from the improved global graph ,in It is represented as a sequence matrix with t rows and t columns, where t represents the maximum length of the sequence. In order to measure the importance of each user u in the improvement graph, that is, the global graph of user behavior sequences, an adaptor is designed in the embodiment of the present invention. The adaptor introduces a personalized local vector for each user. ,in It is represented as a sequence matrix with 1 row and d columns, where d represents the maximum length of the local vector, and is projected into a scalar through a multi-layer perceptron:

[0088]

[0089] in Represented as a local vector The initial representation of Represented as a multi-layer perceptron.

[0090] Then, in order to inject local graph relationships into the sequence representation, we propose to incorporate the user's personalized information into the self-attention mechanism in the form of relative position encoding, as shown in the following formula:

[0091] ;

[0092] in It is expressed as involving the user's personalized information in the self-attention mechanism in the form of relative position encoding. Q, K and V refer to the three input representation vectors used in the self-attention mechanism, Q represents the query vector, K represents the key vector, represents a numeric vector, Represents the softmax operation, specifically a function that converts a set of values into a probability distribution. represents the transpose of the key vector, represents the product of the query vector and the transpose of the key vector, and d represents the maximum length of the local vector.

[0093] The encoder and decoder in the self-attention mechanism process the subgraphs of each user behavior sequence, i.e., the Transformer operation. After the Lth layer of Transformer, the output of the local module is as follows:

[0094]

[0095] Where, Denoted as the local module output of the sequence encoder, Transformer operations represented as local modules, is the historical interaction sequence representation of user u, where u represents the user, .

[0096] It should be explained that the encoder and decoder in the Transformer model process the subgraphs of each user behavior sequence through a self-attention mechanism and a feedforward neural network layer. Specifically, the encoder captures the dependencies and semantic information of the input sequence, while the decoder generates a new symbol sequence based on the encoder output and the previously generated output. This structure makes the Transformer model powerful in processing long sequences and capturing complex dependencies.

[0097] Furthermore, the global graph of user behavior sequences is injected into the global module of the sequence encoder. The specific analysis process is as follows:

[0098] Acquire collaborative information from the global graph of user behavior sequences, where collaborative information specifically refers to shared or related features, patterns, or relationships between different sequences. This collaborative information helps the model understand the intrinsic connections between sequences, thereby improving the modeling and prediction capabilities of sequence data. For example, in an embodiment of the present invention, the purchase history sequences of different users may all contain a preference for a certain type of product. This preference is a global feature, and the collaborative information of the global graph of user behavior sequences can be extracted from the user's behavior logs on the website or application.

[0099] The global module uses a global query matrix shared by all sequences to learn collaborative information about the global graph of user behavior sequences. The output of the global module is constructed as follows:

[0100]

[0101] Where, is the global module output of the sequence encoder, is the softmax operation, is the global Query matrix, is the learnable parameter matrix, is the learnable parameter matrix, is the embedding matrix of the current sequence, as well as Both represent matrices, It is represented as regularization.

[0102] Specifically, the outputs of the local module and the global module are fused to obtain a representation of the user behavior sequence. The specific analysis process is as follows:

[0103] Obtain the position weight of each position point p in the global graph of user behavior sequence. The position weight can be determined based on the importance of the node by using relevant algorithms in graph theory, such as PageRank, to analyze the global graph of user behavior sequence.

[0104] The PageRank algorithm calculates position weights in the following process: for each position point in the global graph of the user behavior sequence, an initial PageRank value is assigned to it. This initial value is usually set to 1 / N, where N is the total number of all position points. This setting is to ensure that each position point has the same starting point at the beginning of the algorithm, which is convenient for subsequent iterative calculations; determine the damping coefficient. The damping coefficient d is a constant between 0 and 1. It represents the probability that the user will continue to execute the current interaction relationship when interacting with the item. Usually d is set to 0.85, which means that the user has an 85% probability of interacting with the item at the current position point, and the remaining 15% probability of randomly jumping to other position points; iterative calculation is performed according to the PageRank calculation formula. The formula is as follows:

[0105]

[0106] Where PR(A) represents the PageRank value of vertex A, d is the damping coefficient, N is the total number of all vertices, T1, T2, ..., Tn are the edges pointing to location point A (i.e., the relationship where A exists), and C(T1), C(T2), ..., C(Tn) are the edges connected to location point A (i.e., how many other locations location point A points to).

[0107] In each iteration, the new PageRank value of each location point is calculated based on the previous PageRank value and the above formula. The iterative calculation will continue until the difference between the PageRank values of all location points in the two previous iterations is less than a predetermined threshold (that is, convergence is reached). This threshold is usually a very small positive number to ensure the stability and accuracy of the algorithm.

[0108] In order to avoid overemphasizing the recent intention, the embodiment of the present invention introduces To measure the importance of the item at position p for predicting user intention, it is constructed as follows:

[0109]

[0110] in

[0111] Where, It is represented as the key vector of position point p, Represented as the query vector of the current position point p, is the global representation, It is a local representation. 、 as well as are all learnable parameters, LN refers to layer normalization, Expressed as an activation function, its mathematical expression is f(x)=max(0,x), that is, for input x, if x is greater than 0, then output x; if x is less than or equal to 0, then output 0.

[0112] In order to fuse the information obtained by modeling the global and local modules to enhance the ability to extract user intent, the output representation of the fusion layer is constructed as follows:

[0113]

[0114] Where, is the output of the fusion layer, is the importance of the item at position p for predicting user intention, Denoted as the local module output of the sequence encoder, is the global module output of the sequence encoder.

[0115] The representation of the user behavior sequence is obtained, specifically The last vector in is used as the representation of user sequence, where .

[0116] S3. Through the sequence encoder, the user intent is extracted from the user behavior sequence, and compared with the representation of the user behavior sequence. The representation of the user behavior sequence is compared with the representation of other user behavior sequences to comprehensively obtain the user's sequence recommendation.

[0117] It should be explained that in order to overcome data sparsity and construct the intention self-supervised learning signal, this embodiment uses a data augmentation method to divide the prefix of the original sequence into a new training sequence. The operation structure is as follows:

[0118]

[0119] in Represented as data augmentation, is the label of each subsequence. In this embodiment, subsequences with the same label are considered to have similar intentions, so subsequences with the same label are divided into the same category.

[0120] In a specific embodiment, it can be expressed as:

[0121] Original sequence: 6410 11091 5596 4981 8566 1996 11045 1976

[0122] After data enhancement: 6410 11091 5596 4981 6410 11091 5596 4981 8566 6410 11091 5596 4981 8566 1996 6410 11091 5596 4981 8566 1996 11045 6410 11091 5596 4981 8566 1996 11045 1976

[0127] Specifically, the representation of the user behavior sequence is compared with the representation of other user behavior sequences. The specific analysis process is as follows:

[0128] User intent can be viewed as collaborative information mined from the collective historical behavior of all users. To mutually promote sequence representation learning and intent learning, this embodiment treats the cluster centers of subsequence representations as intent prototypes.

[0129] First, comparative learning is performed between user behavior sequences to enhance the model's ability to understand and distinguish user intent. Different behavior sequences with the same label may have the same intent. Therefore, the goal of this embodiment is to bring sequences with the same intent closer together in vector space, while simultaneously pulling sequences with different intents further apart.

[0130] Get the current sequence of the user behavior sequence, recorded as , where the current sequence of the user behavior sequence refers to obtaining a subsequence with a specific label or intention in the user behavior sequence, which can be obtained by dividing the user behavior sequence.

[0131] For the current sequence , randomly sample from sequences with the same label to obtain sequences with the same intent , using this as a comparison between sequences, the learning task is constructed as follows:

[0132]

[0133] in

[0134] Where, is the sequence-level contrast loss between user behavior sequences, is the contrast representation of sample pairs, and Represented as two vectors in a vector space, is a pair of positive samples, is the inner product, is the set of users who have the same label as two positive samples in a mini-batch, It is represented as the sequence of users in the user set that do not have the same label.

[0135] In addition, in the embodiment of the present invention, comparative learning is performed between user behavior sequences and user intentions to further collaborate intention learning and behavior learning.

[0136] Furthermore, the user intention is compared with the representation of the user behavior sequence. The specific analysis process is as follows:

[0137] In real life, the intention behind purchasing the same item may vary depending on the situation. After encoding all sequences through the intent encoder that shares parameters with the sequence encoder, this embodiment uses K-means to cluster the output sequence representations to obtain K types of user intentions that reflect the user's interactive behavior decision. The intention is formalized as , and the intentions behind the same behavior may be different.

[0138] Get two vectors of the global graph of user behavior sequence in the vector space, which are recorded as and , this vector represents the position of the user behavior sequence in the vector space.

[0139] In vector space, the vector and The intention prototype with the smallest distance is used as the intention of the two vectors. and .

[0140] The intention with the minimum distance mentioned above, where the distance can be calculated by Euclidean distance, specifically, the square root of the sum of the squares of the differences between each corresponding position of the calculation vector and the intention.

[0141] The contrastive learning task between the representation of user behavior sequences and user intentions is constructed as follows:

[0142]

[0143] Where, Expressed as the intent-level contrast loss between user behavior sequence and user intent, The contrast representation of sample pairs, and Represented as two vectors in a vector space, and Represented as a vector and The corresponding user intention.

[0144] In this embodiment, the sequence representation and the corresponding nearest intent center are regarded as positive pairs, and other intent centers are regarded as negative pairs. At the same time, this embodiment also excludes sequences with the same intent in the small batch to avoid false negative samples.

[0145] S4. Based on user-based sequential recommendation, users are predicted and the sequential recommendation is optimized according to the prediction results, thereby completing the sequential recommendation based on adaptive global graph modeling and intent comparative learning.

[0146] Furthermore, the user-based sequence recommendation is used to predict the user, and the specific analysis process is as follows:

[0147] It should be explained that the next item prediction learning in sequential recommendation can be summarized as a classification task for the entire item set.

[0148] The prediction of the user is specifically a given sequence representation And the item embedding matrix V, the prediction score is calculated as follows:

[0149]

[0150] in is the prediction score, , For a given sequence representation, is the transpose of the item embedding matrix, is the determinant representation of the embedding matrix, Expressed as a normalized function.

[0151] The cross entropy loss is optimized to maximize the probability of correct prediction:

[0152]

[0153] Where, To predict the accuracy of users, Denote as the predicted score of the true item g, is the next real item of interaction of user u, , is the predicted item of the next interaction of user u, , Denote as the prediction score of the predicted item g.

[0154] Specifically, the sequence recommendation is optimized according to the prediction results, specifically by adopting a multi-task training strategy to jointly optimize multiple prediction tasks.

[0155] To enhance the model’s ability to capture global and local information from the graph and learn the user’s primary intent, this embodiment uses a multi-task training strategy to jointly optimize multiple tasks. Specifically, the optimization is based on the joint loss in this embodiment, calculating their linear weighted sum:

[0156]

[0157] in, Expressed as joint loss, is the accuracy of the prediction for the user, is the global information encoding loss, is the sequence-level contrast loss between user behavior sequences, It is the intent-level contrast loss between user behavior sequences and user intent. and They are the hyperparameters that control the loss of each contrastive learning task, specifically those important parameters that affect the calculation and behavior of the loss function, such as the temperature parameter , learning rate, batch size, etc. These hyperparameters are not derived through the training process, but are pre-set before training begins.

[0158] To verify the effectiveness of the embodiments of the present invention, experiments were conducted on three public datasets from the real world: the Beauty dataset, the Sports dataset, and the Toys dataset. These datasets are derived from user review data collected from Amazon, one of the largest e-commerce companies in the United States. Figure 3 Detailed statistics for all datasets are listed in Figure 3 Detailed statistical data table of the data set shown in the figure Represented as the specific number of user sets, It is expressed as the number of item sets, Iterations represents the number of iterations, (Average length / number of users) represents the average value of some metric (such as rating, comment length, browsing time, etc.) of the interaction between users (U) and items (V). (Average length / Number of visitors) represents the average value of user interaction attracted by each item among all items, and Sparsity represents sparsity.

[0159] Depend on Figure 3It can be seen that the three datasets differ in average length and number of interactions, but their sparsity is very high, indicating that there are a large number of missing values or non-interactions in these datasets, so the three datasets need to be preprocessed.

[0160] The above preprocessing,in this embodiment, only uses the “5-core” dataset, excluding inactive users and unpopular items with less than 5 interactions,and adopts the leave-one-out evaluation method to segment the data to evaluate the sequential recommendation model,for example, given the sequence In this embodiment, For training, For verification, For testing purposes.

[0161] In order to comprehensively evaluate the proposed model and baseline, this example uses the top K hit rate (HR@K) and top K normalized discounted cumulative gain (NDCG@K), which are widely used in sequential recommendation, as evaluation indicators of recommendation performance, where HR measures the accuracy of real items appearing in the top K recommendations, and NDCG is a ranking loss that evaluates the position of real items in the top K recommendations. Generally speaking, higher HR@K and NDCG@K metrics indicate better model performance.

[0162] To demonstrate the effectiveness of the present invention, we compared the performance of the proposed method with 13 popular recommendation models, covering a variety of recommendation methods. These methods can be divided into four categories: (1) non-sequential models; (2) general sequential models; (3) sequential models based on self-supervised learning; and (4) intent-based sequential models. Detailed information about the baselines is as follows:

[0163] BPR: We propose a general Bayesian optimization criterion and learning algorithm for personalized ranking from implicit feedback.

[0164] GRU4Rec: This model uses gated recurrent units to represent user behavior sequences.

[0165] Caser: This method uses convolutional neural networks to extract sequential patterns from recent interactions as local features.

[0166] SASRec: adopts the self-attention mechanism to capture dynamic interest patterns in user behavior sequences, considering the information of the entire sequence when processing the data at each time step.

[0167] BERT4Rec: This model utilizes bidirectional self-attention to model user sequences.

[0168] : It uses the mutual information maximization principle to learn the association between different data. Our experiments only use the masked item prediction (MIP) task.

[0169] CL4SRec: This model incorporates contrastive learning into sequential recommendation to alleviate the problem of data sparsity.

[0170] CoSeRec: This model proposes novel data augmentation techniques to alleviate the cold start problem.

[0171] DuoRec: This method utilizes dropout to better preserve the semantics of augmented data, thereby improving the distribution of item embeddings.

[0172] MCLRec: This framework introduces learnable model enhancement and proposes a meta-update method.

[0173] ICLRec: This model learns the user's intent distribution function and incorporates these intents into the sequence model using self-supervised learning.

[0174] IOCRec: This method aims to identify the user’s main intent from multiple intents to reduce noise.

[0175] SparseEnNet: This model adopts adversarial learning and clustering-based self-training methods to enhance recommendation performance.

[0176] The overall performance of all baselines was compared with the method of the embodiment of the present invention. The comparison results are as follows: Figure 4 As shown, Figure 4 The performance comparison results comparison chart shows the main experimental data.

[0177] Depend on Figure 4 , we can draw the following conclusions:

[0178] (1) Sequential patterns are indispensable for recommendation systems. GRU4Rec and Caser, which integrate user sequence information, both outperform the non-sequential method BPR in recommendation performance. This shows that sequential information is helpful in capturing users' dynamic preferences. In addition, the Transformer-based SASRec further improves performance compared to traditional models, demonstrating the strength of the attention mechanism in extracting sequential features.

[0179] (2) SSL-based sequence models generally outperform general sequence methods. Although BERTRec and The performance on some indicators is slightly lower than SASRec, but this difference may be due to the mismatch between the objectives of the cloze task and the next item prediction task. Models based on the self-supervised learning paradigm effectively alleviate the data sparsity problem in sequence recommendation. CL4SRec and CoSeRec further effectively capture the deep characteristics of user behavior through data augmentation methods and contrastive learning paradigms. One possible reason for DuoRec's better performance compared to CoSeRc is that it regards sequences with the same next item in contrastive learning as positive samples, which enhances the distinction of sequence semantics.

[0180] (3) Intent-oriented sequence models show improvements over most SSL-based models such as CL4SRec and CoSeRc. By comparing ICLRec and CL4SRec, it can be observed that incorporating intent can improve performance. The comparison between ICLRec and IC4SRec shows that incorporating global graph information can further improve performance. ICLRec, IOCRec, and SparseEnNet all construct contrastive self-supervised learning between user representations and their corresponding intent representations to improve the quality of user representations and thus improve recommendation performance. The reason why they are not as effective as MCLRec may be that the noise introduced by random data augmentation disturbs the original intent of the sequence. The method proposed in this paper avoids this situation.

[0181] (4) By comparing the proposed IC4SRec with all baselines, it can be found that IC4SRec outperforms existing models on all datasets. Compared with the best baseline, IC4SRec improves by 8.84% to 28.66% on HR and by 29.15% to 48.01% on NDCG. The reasons can be summarized as follows: (1) IC4SRec extracts global collaborative information from the refined graph to improve sequence encoding, producing higher quality user representations compared to ICLRec, IOCRec, and SparseEnNet. (2) Our method alleviates the false negative problem while maintaining a high degree of semantic consistency between the sequence and its corresponding intent through a contrastive learning paradigm, further improving the robustness and performance of the model.

[0182] In order to further analyze the impact of global collaborative information and contrastive learning on embedding learning, we compare the visualization results of the embedding matrices learned by different models on the Beauty and Toys datasets, as shown in the figure below. Figure 5 As shown, the color bar on the right represents the frequency of items in the dataset. This comparison involves ICLRec and the method proposed in this embodiment.

[0183] Depend on Figure 5Comparing the visualization results, we can see that in Figures (a) and (c), ICLRec's embeddings exhibit a relatively narrow and concentrated conical distribution in the latent space. This indicates that ICLRec tends to focus on high-frequency items, resulting in poorly balanced item representation. Rare items may be marginalized and thus ineffectively recommended to users. The concentrated distribution of points indicates that ICLRec has difficulty distinguishing between high-frequency and low-frequency items, resulting in insufficient generalization across the entire item spectrum.

[0184] In contrast, Figures (b) and (d) show the embedding distributions generated by our model, which exhibit a more balanced distribution across high- and low-frequency items. The embeddings cover a wider range in the latent space, indicating that our model effectively utilizes contrastive learning and global information to provide a more uniform distribution. This balanced embedding distribution demonstrates that our model is able to capture subtle relationships between items, thereby improving performance on sequential recommendation tasks. The color bar in each figure represents the frequency of the item, further illustrating the advantage of our model over ICLRec when handling rare items.

[0185] Overall, these visualizations show that the model proposed in this example generates higher-quality item embeddings, characterized by a more even distribution of items in the latent space. This balanced distribution aligns with the goal of contrastive learning and effectively improves the performance of the recommendation system, enabling it to better handle both popular and rare items.

[0186] The above content is merely an example and explanation of the structure of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the present invention, they should all fall within the scope of protection of the present invention.

Claims

1. A sequence recommendation method based on adaptive global graph modeling and intent contrastive learning, characterized by: include: S1. Obtain historical interaction data between users and items, sort them based on rules, and obtain a user historical behavior sequence. This is then constructed into a correlation graph, which is then improved and optimized to obtain and record a global user behavior sequence graph. S2. Divide the global graph of user behavior sequences into subgraphs of each user behavior sequence, combine them with the user embedding, and inject them into the local module of the sequence encoder. Then, inject the global graph of user behavior sequences into the global module of the sequence encoder, fuse the outputs of the local and global modules, and obtain a representation of the user behavior sequence. S3. Extract user intent from the user behavior sequence through a sequence encoder, perform comparative learning on the representation of the user behavior sequence, and compare the representation of the user behavior sequence with the representation of other user behavior sequences to comprehensively obtain sequence recommendations for the user. S4. User-based sequential recommendation: predict users and optimize the sequential recommendation based on the prediction results, thereby completing sequential recommendation based on adaptive global graph modeling and intent comparative learning; The rule-based sorting is performed to obtain the user's historical behavior sequence, which is constructed into a correlation graph. The specific analysis process is as follows: The rule is specifically to sort by the time sequence of each historical interaction time point; The historical interaction data between the user and the item, specifically the interaction data between the user and the item at each historical interaction time point; The user historical behavior sequence is obtained by sorting the interaction data between the user and the item at each historical interaction time point in chronological order, obtaining and recording each user historical behavior subsequence. The collection of each user historical behavior subsequence is called the user historical behavior sequence. The user's historical behavior sequence is divided into a set of items and a set of relationships between users and items; The construction is an association graph, specifically connecting the set of items in the user's historical behavior sequence and the set of relationships between users and items in a predefined manner. The specific expression is: Association diagram , where V is the set of items, is the set of relationships between users and items, It is represented as the relationship between vertex i and vertex j of the e-th edge in the association graph, .

2. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The association graph is improved and optimized to obtain and record the global graph of user behavior sequence. The specific analysis process is as follows: The above-mentioned improvement and optimization of the association graph is to optimize the association graph through a lightweight graph convolutional network as a graph encoder, and obtain and record it as a global graph of user behavior sequence; The global graph representation of the user behavior sequence is obtained by embedding the weighted sum of the association graphs from each L layer of the lightweight graph convolutional network through the graph encoder, as shown below: ； Where, It is a global graph representation of user behavior sequence, is the association graph representation of layer 0, Represented as the association graph representation of the rth layer, is the association graph representation of the Lth layer, L is a specific layer representation in the lightweight graph convolutional network, ,r represents any layer in the lightweight graph convolutional network.

3. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The global graph of user behavior sequences is divided into subgraphs of user behavior sequences, combined with user embedding, and injected into the local module of the sequence encoder. The specific analysis process is as follows: Obtain personalized information about users based on the global graph of user behavior sequences; The above is combined with user embedding and injected into the local module of the sequence encoder. Specifically, the user's personalized information is included in the self-attention mechanism in the form of relative position encoding of the global graph of the user behavior sequence. The encoder and decoder in the self-attention mechanism process each subgraph of the user behavior sequence, that is, the Transformer operation. After the Lth layer of Transformer, the output of the local module is as follows: ； Where, Denoted as the local module output of the sequence encoder, Transformer operations represented as local modules, is the historical interaction sequence representation of user u, where u represents the user, .

4. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The global graph of user behavior sequences is injected into the global module of the sequence encoder. The specific analysis process is as follows: Obtain collaborative information of the global graph of user behavior sequences; The global module uses a global query matrix shared by all sequences to learn collaborative information about the global graph of user behavior sequences. The output of the global module is constructed as follows: ； Where, is the global module output of the sequence encoder, is the softmax operation, is the global Query matrix, is the learnable parameter matrix, is the learnable parameter matrix, is the embedding matrix of the current sequence, as well as Both represent matrices, It is represented as regularization.

5. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 4, characterized in that: The outputs of the local module and the global module are fused to obtain the representation of the user behavior sequence. The specific analysis process is as follows: Get the position weight of each point p in the global graph of user behavior sequence, denoted as ; The output representation of the fusion layer is constructed as: ； Where, is the output of the fusion layer, is the importance of the item at position p for predicting user intention, Denoted as the local module output of the sequence encoder, is the global module output of the sequence encoder; The representation of the user behavior sequence is obtained, specifically The last vector in is used as the representation of user sequence, where .

6. The sequence recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The user intention is compared with the representation of the user behavior sequence. The specific analysis process is as follows: Get two vectors of the global graph of user behavior sequence in the vector space, which are recorded as and ; In vector space, the vector and The intention prototype with the smallest distance is used as the intention of the two vectors. and ; The contrastive learning task between the representation of user behavior sequences and user intentions is constructed as follows: ； Where, It is represented as a contrastive learning between the representation of user behavior sequence and user intention, Represented as a sequence of user behaviors, and Represented as a vector in a vector space, and Expressed as and The intent prototype with the smallest distance is taken as the user intent.

7. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 6, characterized in that: The representation of the user behavior sequence is compared with the representation of other user behavior sequences. The specific analysis process is as follows: Get the current sequence of the user behavior sequence, recorded as ; For the current sequence , randomly sample from sequences with the same label to obtain sequences with the same intent , using this as a comparison between sequences, the learning task is constructed as follows: ； Where, For the comparison between user behavior sequences, is the user behavior sequence, and Represented as two vectors in a vector space.

8. The sequence recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The user-based sequence recommendation is used to predict the user. The specific analysis process is as follows: The prediction of the user is specifically a given sequence representation And the item embedding matrix V, the prediction score is calculated as follows: ； in is the prediction score, , For a given sequence representation, is the transpose of the item embedding matrix, is the determinant representation of the embedding matrix, Expressed as a normalized function; The cross entropy loss is optimized to maximize the probability of correct prediction: ； Where, To predict the accuracy of users, is the next real item of interaction of user u, is the predicted item of the next interaction of user u, .

9. The sequential recommendation method based on adaptive global graph modeling and intent contrastive learning according to claim 1, characterized in that: The sequence recommendation is optimized according to the prediction results, specifically by adopting a multi-task training strategy to jointly optimize multiple prediction tasks.

Images