Bidirectional fine tuning method for large recommendation model with efficient parameters fused with collaborative information

Abstract

The invention provides a parameter-efficient collaborative information-fused big recommendation model bidirectional fine tuning method, which comprises the following steps of: obtaining an input text, and normalizing the input text; designing an MoE architecture comprising a plurality of query experts to process different types of users, and integrating specific cooperation information of the users into a normalized text; and training parameters in the normalized text, and selecting the item with the highest score as the next item to be recommended to the user. According to the method, the large recommendation model can be adapted to an effective recommendation system in a parameter efficient mode, and the method is obviously superior to the most advanced method.

Description

Technical Field

[0001] The present invention belongs to the field of big data recommendation, and specifically relates to a parameter-efficient bidirectional fine-tuning method for a large recommendation model integrating collaborative information. Background Art

[0002] Sequential recommender systems aim to learn an effective representation of user interests based on their past interactions and recommend future items that the user is most likely to interact with. Due to their ability to capture the dynamic nature of user preferences and effectively improve user satisfaction, sequential recommender systems are widely used in various scenarios such as e-commerce, streaming services, and social media platforms.

[0003] In traditional sequential recommender systems, items are mainly represented by unique IDs. Various methods have been adopted to obtain effective ID embeddings based on user interaction sequences, including Markov chains, RNN / CNN models, and self-attention models. Although ID-based methods are promising in capturing the potential associations between users and items, they do not take into account the rich semantic information contained in the text descriptions of items (e.g., item titles), resulting in unsatisfactory performance. To address this issue, efforts have been made to encode item semantic information using language models. However, previous works have mainly focused on small and medium-sized language models, which have limited performance.

[0004] Recently, large language models (LLMs) have made significant progress in language understanding. It is a promising approach to leverage the powerful semantic information modeling capabilities of LLMs pre-trained on a large text corpus to capture the semantic information of items. There are two main paradigms in existing work to integrate LLMs into recommendation tasks. The first paradigm is to use LLMs to make recommendations directly in the form of natural language. These works design special prompts or use supervised fine-tuning to let LLMs answer given recommendation questions. However, this paradigm can only determine the recommendation of one item at a time, and the usage frequency of LLMs increases linearly with the number of candidate items. Therefore, these methods are typically only used in the re-ranking stage that contains only dozens of candidate items. The second paradigm is to use LLMs as encoders to provide item / user embeddings for similarity comparison and next item prediction. As Figure 1 As shown in

[15] , given a user interaction history represented in natural language, these works use LLM to encode each token in the input text, and then perform various pooling strategies on the output token embedding to derive user embeddings. Although these works are promising, they usually require training a large number of parameters and require a lot of computing resources. In addition, these works have difficulty in effectively integrating ID-based collaboration information into LLM, which affects the effectiveness of recommendation. Although efforts have been made to map collaboration embeddings into the language space of LLM using simple linear projections, these methods fail to take into account the diverse characteristics of different types of users, which may lead to poor recommendation results. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a parameter-efficient bidirectional fine-tuning method for a large recommendation model integrating collaborative information.

[0006] The technical solution of the present invention is as follows:

[0007] A parameter-efficient bidirectional fine-tuning method for a large recommendation model integrating collaborative information, the method comprising:

[0008] Get the input text and normalize it;

[0009] Design a MoE architecture with multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text;

[0010] Train the parameters in the normalized text and select the item with the highest score as the next item to recommend to the user.

[0011] Furthermore, the normalized text includes a prefix and a suffix; the prefix is ​​used to capture collaborative information and adapt the large model to the recommendation task; the suffix is ​​used to capture information of the entire input text and convert it into an embedded representation required by the recommendation system.

[0012] Furthermore, a MoE architecture with multiple query experts is designed to handle different types of users and integrate user-specific collaboration information into the normalized text. Specifically:

[0013] Design a MoE architecture that includes multiple query experts, each of which is responsible for processing a specific type of collaborative information and interacting with the collaborative information through Transformer blocks;

[0014] Use a router to calculate the scores of different query experts based on the user interaction history, and select the expert with the highest score to handle the current user;

[0015] The selected query experts interact with the collaborative information through the Transformer block.

[0016] Furthermore, the expert's score is obtained by the following formula:

[0017]

[0018] Among them, r j (u) represents the score of expert j for user u, p i,j (u) represents the preference or relevance of user u to expert j in the i-th interaction, and N is the total number of user interactions.

[0019] Furthermore, the interacting with the collaborative information through the Transformer block is specifically interacting with the internal state of the expert network through a self-attention mechanism and a cross-attention mechanism.

[0020] Furthermore, it also includes training through a loss function:

[0021]

[0022] Among them, λ is a hyperparameter, For the loss of the main task, Loss of auxiliary tasks.

[0023] Furthermore, the parameters in the normalized text after training are specifically:

[0024] Freeze the parameters of the recommendation model, calculate the similarity between the user embedding and item embedding obtained by the recommendation model, and select the item with the highest score as the next item to recommend to the user.

[0025] Furthermore, the calculation of user embedding and item embedding is specifically as follows:

[0026]

[0027] Among them, s(u,i)∈R represents the probability that item i will become the next browsed item of user u; represents the dot product of user embedding and item embedding, ||h u || and ||h i || represents the Euclidean norm of user embedding and item embedding respectively;

[0028] To predict the next item, we iterate over each item i in the item set and select the item i^ with the highest score as the next item:

[0029]

[0030] Where I represents the item set, and argmax represents selecting the item i that maximizes the similarity score s(u,i).

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] The present invention adapts LLM to sequential recommendation through trainable prefixes and suffixes. The prefix adapts LLM to recommendation tasks with collaborative information, while the suffix converts the LLM output from the language space to the recommendation space and obtains high-quality user / item embeddings. In order to effectively integrate ID-based collaborative information for more accurate recommendations, M-Former, a lightweight MoE-based query Transformer, is also introduced, which uses a group of query experts to capture the diverse collaborative features of different types of users. Finally, a multi-task loss function and a two-stage training strategy are adopted to train Laser for sequential recommendation. Extensive experiments on real-world datasets show that the present invention is able to adapt LLM to an effective recommendation system in a parameter-efficient manner, significantly outperforming the state-of-the-art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the description and claims, serve to illustrate the embodiments of the invention. Where appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0034] Figure 1 A schematic diagram of the method framework of the present invention is shown;

[0035] Figure 2 A schematic diagram showing the performance comparison of the Scientific dataset of the present invention under a zero-sample setting and a low-resource setting is shown. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] The embodiment of the present invention provides a parameter-efficient bidirectional fine-tuning method for a large-scale recommendation model integrating collaborative information.

[0038] First, we describe how to convert the user interaction history Du={Di1, Di2, ..., Di N} into a coherent text Tu = {t1, t2, ..., t W}, as the input of LLM for sequence recommendation, and then we introduce how to adapt LLM to the recommendation task through the proposed parameter-efficient bidirectional fine-tuning method.

[0039] The user interaction history Du is organized into the input text Tu of LLM for recommendation using a unified template. For example, given a user who has browsed "Kaytee Aspen Bedding Bag", "Guitar A-Frame SupportS" and "KONG Wubba DogToy", the present invention can normalize the corresponding input text Tu into the following form.

[0040] Please summarize the user's characteristics into a token in chronological order based on the browsing history. The items that the user has browsed are as follows:

[0041] >>1.Kaytee Aspen Bedding Bag (Brand: Kaytee, Category: Kaytee)

[0042] >>2.Guitar A-Frame Support (Brand: Sageworks, Category: Sageworks) ...

[0044] >>4.KONG Wubba Dog Toy (Brand: KONG, Category: KONG)

[0045] Using the designed template, LLM can summarize the user browsing history into a token (i.e., a suffix attached to the end of the input text) according to the instructions in the input text, and its corresponding output embedding can be used as the user embedding hu for similarity comparison with the item embedding and next item recommendation. In order to obtain the item embedding, the present invention also uses the same template as mentioned above. In particular, for a specific item i, the present invention regards it as a special user interaction history, which only contains this item. Therefore, the present invention uses the same template to formulate the input text of LLM, and uses the output attached suffix embedding as the item embedding hi. In this way, the present invention can obtain the embedding of each item i in the item set I, and the original user-item similarity comparison for recommendation can be regarded as a special user-user similarity comparison. The advantage of this is that the unified template can minimize the impact of hard templates on the performance of LLM.

[0046] Given an input text Tu = {t1, t2, ..., t W}, it will be expanded to trainable prefixes and suffixes:

[0047]

[0048] where P = {p1, p2, ..., p L} refers to a prefix containing L preceding virtual tokens, and s refers to a suffix consisting of one additional virtual token. During model training, the present invention freezes the parameters of the LLM and customizes them for the recommendation task by optimizing the trainable virtual tokens added to the prefix P and the suffix s, which greatly reduces the size of the parameters that need to be trained.

[0049] The prefix P containing L dummy tokens is responsible for adapting the LLM to the recommendation task through collaborative information. These dummy tokens can act as placeholders, allowing the LLM to capture task-specific information during fine-tuning.

[0050] In addition to the prefix P, the present invention also appends a special trainable virtual token s, called the suffix, to the end of the input text Tu. Previous works have attempted to obtain user embeddings by average pooling the token embeddings output by LLM. However, most generative LLMs are based on masked attention mechanisms, which means that only the last token can observe the entire input. Therefore, these works may introduce noise by averaging all output embeddings. In this work, the present invention uses an additional trainable virtual token s to capture the information of the entire input Tu, and its output embedding hs can be used as the user embedding hu for similarity comparison and next item prediction. Formally, the encoding process of LLM can be expressed as:

[0051]

[0052] Where e∈Rd represents the input token embedding of input Tu={p1,...,t1,...,s}, h∈Rd represents the corresponding output embedding, and d represents the hidden size of LLM. Through the trainable suffix s, the present invention can effectively transform the output embedding of LLM from language space to recommendation space. When the user interaction history or a single item is taken as the input of LLM, the present invention can directly use the output suffix embedding hs as the user embedding hu or the item embedding hi for further recommendation.

[0053] Given the user embedding hu∈Rd and item embedding hi∈Rd obtained from LLM, the present invention can calculate the similarity between them as follows:

[0054]

[0055] Where s(u,i)∈R represents the probability that item i will become the next item browsed by user u. In order to predict the next item, the present invention traverses each item i in the item set I and selects the item i^ with the highest score as the next item:

[0056]

[0057] The present invention uses trainable prefix P and suffix s to adapt LLM to recommendation. In order to achieve better recommendation results, the present invention incorporates collaborative information through prefix P. Existing work attempts to project the collaborative embedding encoded by the ID-based sequential recommender system into the language space of LLM using a unified linear layer. However, this approach is too simple and cannot detect the diverse characteristics of different types of users. To address this challenge, the present invention introduces M-Former, a query Transformer based on MoE, which uses a group of query experts to handle different types of users and integrates user-specific collaborative information into prefix P.

[0058] like Figure 2 As shown in Figure 1, there are K query experts handling different types of users, and each expert contains L trainable virtual tokens. In order to select the most appropriate expert to handle user-specific collaborative information, the present invention sets up a router to calculate the scores of different experts given a specific user. Formally, given the user interaction history Su of user u = {i1, i2, ..., i N}, the corresponding project ID sequence IDu={idi1,idi2,...,idi N} is used as the input of the pre-trained ID-based sequential recommender system (frozen) and is encoded as Cu∈RN×di, where N represents the length of the interaction history Su and di represents the hidden size of the sequential recommender system. Then, Cu is fed into the router, which is implemented by a fully connected layer in this work. Based on the N user interaction items embedded as Cu, the matching degree of the K query experts can be calculated as:

[0059]

[0060] Where Wr∈RK×di is the linear weight of the router and m(u)∈RN×K. Then, the score of the i-th item to the j-th expert can be calculated as:

[0061]

[0062] And the final score of the j-th query expert can be obtained by:

[0063]

[0064] get.

[0065] Finally, the query expert with the highest score will be selected to handle the specific user u.

[0066] like Figure 1As shown, the selected query expert containing L virtual tokens is sequentially fed into Z Transformer blocks to interact with the collaborative information Cu. In this way, the query expert integrates the collaborative information into its L trainable virtual tokens, which are further used as the above prefix P = {p1, p2, ..., pL} to adapt LLM to sequential recommendation while enhancing the collaborative information.

[0067] Formally, the query expert can be represented as E∈RL×dm, where dm is the hidden size of the M-Former. In the Transformer block, E is first encoded through a self-attention layer and then projected into the query matrix Q for interaction with the key / value matrix (K / V) projected from the ID-based collaborative embedding Cu in the cross-attention layer. In this way, the present invention updates the embedding E of the query expert and integrates the ID-based collaborative information into it. Then, through a linear projection layer set on top of the Z Transformer blocks, the query expert is projected to the hidden size d of the LLM and used as a prefix P={p1,p2,...,p L} to adapt LLM to sequence recommendation.

[0068] A multi-task training strategy is proposed to jointly train the proposed Laser for LLM-based sequence recommendation.

[0069] The first training task is the item-item contrast (IIC) task, which is widely used for next item prediction. Following previous work, we use the true next item as the positive example and all other items in item set I as negative examples. Formally, the item-item contrast loss is calculated as:

[0070]

[0071] The second training task is a load balancing task, which is used to encourage load balancing between different query experts of M-Former. As demonstrated in previous work, this task can force the router to assign users with different collaborative characteristics to different query experts, so that each expert can be trained to obtain the best collaborative information integration effect for its user group. Formally, the load balancing loss is calculated as:

[0072]

[0073] Where K is the number of query experts, f j represents the proportion of projects assigned to the jth expert and can be calculated as:

[0074]

[0075] Where N is the number of items in the user's interaction history.j represents the probability ratio assigned by the router to the jth expert, which can be calculated as:

[0076]

[0077] In general, the loss function used by the inventors in this work is:

[0078]

[0079] Where λ is a hyperparameter that controls the weights of different tasks.

[0080] Recommendations are made based on user-item embedding similarity comparisons. Since item embeddings are determined by corresponding trainable suffixes, they will change after different training cycles. The present invention adopts a two-stage training strategy, first finding the most appropriate parameter weights to obtain high-quality item embeddings, and then further training Laser based on these fixed item embeddings to achieve the best recommendation results.

[0081] Specifically, in the first training phase, at the beginning of each cycle, the item embedding is updated using the current weight A to The Laser is then trained on I and validated at the end of the epoch based on the updated parameter weights A'. At the end of the first training phase, the embedding of the best performing epoch is selected and the corresponding parameter weights For the second training phase. In the second training phase, Laser is used Initialize and then train for multiple epochs to further adapt the fixed embedding Finally, the parameter weights that produce the best validation performance is retained in and The test results performed on the above represent the final performance of the Laser.

[0082] To evaluate the effectiveness of the present invention, the present invention conducts experiments on three categories of Amazon review datasets, including "Industry and Science", "Arts, Crafts and Sewing", and "Pet Supplies". Following previous work, the present invention uses the five-core dataset provided by the data source and filters out items with missing titles. Then, the present invention collects interactions from different users and sorts the interacted items in ascending order by timestamp. The statistical information of the preprocessed dataset is shown in Table 1. For item semantic information modeling, the present invention selects item attributes including title, category, and brand.

[0083] Table 1 Statistics of the dataset after preprocessing.

[0084]

[0085] Next, the present invention first describes the MoE strategy, i.e., how to select the most appropriate query expert from a set of experts according to the collaborative features of a specific user. The present invention then explains how the selected query expert interacts with the collaborative information encoded by the frozen ID-based sequential recommender system in the query transformer.

[0086] The present invention is compared with many state-of-the-art baselines, including six traditional methods (SASRec, BERT4Rec, RecGURU, FDSA, ZESRec, RECFORMER) and three LLM-based methods (LLM4REC, ZESRec, LlamaRec). In addition, the frozen ID-based sequence recommender used in the present invention is the pre-trained BERT4Rec, and the frozen LLM used is ChatGLM2-6B. Other trainable modules are randomly initialized.

[0087] Following previous work, we use three popular metrics including Recall@N, NDCG@N, and MRR, where N is set to 10. For data splitting, we adopt the leave-one-out method, where the most recent item in the interaction history is used for testing, the second most recent item is used for validation, and the remaining items are used for training. We consider all items in the item set as candidate items and report the average results on the test data.

[0088] Table 2 Performance comparison of different methods

[0089]

[0090] Table 3 Comparison of parameter scales of different LLM tuning methods

[0091]

[0092] Comparisons are made with nine state-of-the-art baselines on three Amazon datasets. From the experimental results, we can make the following observations.

[0093] First, compared with other excellent sequential recommendation methods, the present invention has achieved significant improvements in all indicators of all datasets. For example, on the Pet dataset, compared with the second best method, the present invention improves Recall@10, NDCG@10 and MRR by about 11.37%, 13.27% and 10.63%, respectively. This shows that the framework proposed in the present invention can successfully adapt LLM to an effective sequential recommender system. The present invention believes that the present invention benefits from the Bi-Tuning method, which effectively adapts LLM to sequential recommendation with collaborative information. In addition, when integrating collaborative information, the designed M-Former (MoE-based Query Transformer) captures the diverse characteristics of different types of users for more accurate recommendations.

[0094] In addition, the present invention compares the parameter scale of different LLM tuning methods in Table 3. The present invention can observe that the Bi-Tuning proposed by the present invention is more parameter efficient, containing only 0.135M trainable parameters. The results show that Laser can greatly reduce the scale of trainable parameters and achieve effective LLM adaptation through the proposed Bi-Tuning method, which is better than the SOTA baseline. Please note that in Table 3, the present invention only lists the trainable parameter scale of different LLM tuning methods. The total number of trainable parameters of Laser is about 183.3M, which is also significantly less than 3% of the number of parameters of the LLM backbone ChatGLM2-6B.

[0095] To further demonstrate the effectiveness of the present invention, experiments are conducted to examine its performance in zero-shot and low-resource scenarios. Specifically, the present invention (using both item semantic information and ID-based collaboration information) is compared with two other types of baselines, including BERT4Rec using only ID-based collaboration information and RECFORMER using only item semantic information. The present invention first pre-trains these methods (except ID-based BERT4Rec) on the Pet dataset, and then tests whether they can perform well on another domain with no / limited training data.

[0096] from Figure 2It can be observed that: (1) Laser performs best in zero-shot scenarios. Compared with other baselines, Laser achieves significantly better performance (Recall@10 reaches 0.97, NDCG@10 reaches 0.58), even though it has not seen any items on the Scientific dataset. The present invention attributes this superior performance to the design of the Bi-Tuning framework of the present invention, which fully utilizes the generalization ability of LLM and effectively adapts LLM to sequence recommendation. (2) The present invention only needs to use 5% of the training data to exceed the effect of other baselines that use 100% training data. Compared with the other two baselines, as the proportion of training data increases to 5%, the performance of Laser can quickly rise to a very impressive level. This means that the present invention only needs a very small amount of training data and training time to transfer Laser trained on one domain to another unseen domain, and is accompanied by better results than other baselines that require more training data. This shows that the framework proposed by the present invention can effectively transform LLM into a generalizable sequence recommender system.

[0097] In order to demonstrate the effectiveness of each module in the present invention, the present invention conducts an ablation study and provides the results in Table 4. The present invention can observe that: (1) The experimental results on the two datasets are consistent. Removing any module will cause the performance of Laser to drop significantly. (2) Without MoE, Recall@10, NDCG@10 and MRR drop by 8.28%, 8.63% and 10.92% on average, respectively, indicating that the introduction of MoE can help the framework of the present invention handle the diverse collaborative features of different types of users, resulting in higher quality recommendation results. In addition, without M-Former, these three indicators drop by 9.15%, 13.35% and 16.51% on average, respectively. This shows the importance of using ID-based collaborative information for more accurate recommendations, and the M-Former of the present invention can effectively integrate collaborative information into LLM. (3) Without prefix, Recall@10, NDCG@10 and MRR drop significantly by 20.89%, 24.97% and 25.39% on average, respectively, indicating the important role of prefix in adapting LLM to recommendation tasks. (4) The effectiveness of Laser decreases when any training stage is removed. Specifically, without the first training stage, Recall@10, NDCG@10, and MRR decrease by an average of 49.24%, 47.29%, and 48.28%, respectively, indicating that it is necessary to find appropriate parameter weights to obtain high-quality item embeddings. In addition, without the second training stage, the indicators also decrease by 9.13%, 7.28%, and 9.96%, respectively. This shows that after obtaining suitable item embeddings, it is necessary to continue training to make Laser better adapt to the fixed item embeddings and achieve the best recommendation results.

[0098] Table 4 Results of the ablation study.

[0099]

[0100] The present invention adapts LLM to sequential recommendation through trainable prefixes and suffixes. The prefix adapts LLM to recommendation tasks with collaborative information, while the suffix transforms the LLM output from the language space to the recommendation space and obtains high-quality user / item embeddings. In order to effectively integrate ID-based collaborative information for more accurate recommendations, M-Former, a lightweight MoE-based query Transformer, is also introduced, which uses a group of query experts to capture the diverse collaborative features of different types of users. Finally, a multi-task loss function and a two-stage training strategy are adopted to train Laser for sequential recommendation. Extensive experiments on real-world datasets show that Laser is able to adapt LLM into an effective recommendation system in a parameter-efficient manner, significantly outperforming state-of-the-art methods.

[0101] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A parameter-efficient bidirectional fine-tuning method for recommendation large models integrating collaborative information, characterized in that: The method comprises: Get the input text and normalize it; Design a MoE architecture with multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text; Train the parameters in the normalized text and select the item with the highest score as the next item to recommend to the user.

2. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 1 is characterized in that: The normalized text includes a prefix and a suffix; the prefix is ​​used to capture collaborative information and adapt the large model to the recommendation task; the suffix is ​​used to capture the information of the entire input text and convert it into an embedded representation required by the recommendation system.

3. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 1 is characterized in that: Design a MoE architecture with multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text. Specifically: Design a MoE architecture that includes multiple query experts, each of which is responsible for processing a specific type of collaborative information and interacting with the collaborative information through Transformer blocks; Use a router to calculate the scores of different query experts based on the user interaction history, and select the expert with the highest score to handle the current user; The selected query experts interact with the collaborative information through the Transformer block.

4. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 3 is characterized in that: The expert's score is obtained by the following formula: Among them, r j (u) represents the score of expert j for user u, p i,j (u) represents the preference or relevance of user u to expert j in the i-th interaction, and N is the total number of user interactions.

5. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 3 is characterized in that: The interaction with the collaborative information through the Transformer block is specifically to interact with the internal state of the expert network through the self-attention mechanism and the cross-attention mechanism.

6. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 1 is characterized in that: It also includes training through loss functions: Among them, λ is a hyperparameter, For the loss of the main task, Loss of auxiliary tasks.

7. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 1 is characterized in that: The parameters in the training normalized text are as follows: Freeze the parameters of the recommendation model, calculate the similarity between the user embedding and item embedding obtained by the recommendation model, and select the item with the highest score as the next item to recommend to the user.

8. The parameter-efficient and collaborative information-integrated recommendation large model bidirectional fine-tuning method according to claim 7 is characterized in that: The calculation of user embedding and item embedding is specifically as follows: Among them, s(u,i)∈R represents the probability that item i will become the next browsed item of user u; represents the dot product of user embedding and item embedding, ||h u || and ||h i || represents the Euclidean norm of user embedding and item embedding respectively; To predict the next item, we iterate over each item i in the item set and select the item i^ with the highest score as the next item: Where I represents the item set, and argmax represents selecting the item i that maximizes the similarity score s(u,i).

Images