A parameter-efficient method for fine-tuning large language models with cross-lingual and cross-domain knowledge transfer

Abstract

This invention provides a parameter-efficient method for bidirectional fine-tuning of a large recommendation model that integrates collaborative information. The method includes: acquiring and normalizing the input text; designing a MoE architecture with multiple query experts to handle different types of users and integrating user-specific collaborative information into the normalized text; training parameters in the normalized text; and selecting the highest-scoring item as the next item to recommend to the user. This invention can adapt a large recommendation model into an effective recommendation system in a parameter-efficient manner, significantly outperforming state-of-the-art methods.

Description

Technical Field

[0001] This invention belongs to the field of big data recommendation, specifically involving a bidirectional fine-tuning method for a large recommendation model that integrates collaborative information with high parameter efficiency. Background Technology

[0002] Sequence recommendation systems aim to learn effective representations of user interests based on past interactions and suggest future items that users are most likely to interact with. Because they can capture the dynamic nature of user preferences and effectively improve user satisfaction, sequence recommendation systems are widely used in various scenarios such as e-commerce, streaming services, and social media platforms.

[0003] In traditional sequence recommendation systems, items are primarily represented by unique IDs. Various methods have been employed to obtain effective ID embeddings based on user interaction sequences, including Markov chains, RNN / CNN models, and self-attention models. While ID-based methods show promise in capturing potential associations between users and items, they fail to account for the rich semantic information contained in the item text descriptions (e.g., item titles), leading to suboptimal performance. To address this issue, efforts have been made to encode the semantic information of items using language models. However, previous work has primarily focused on small to medium-sized language models, which have limited performance.

[0004] Recently, large language models (LLMs) have made significant progress in language understanding. Leveraging the powerful semantic information modeling capabilities of LLMs pre-trained on large text corpora to capture the semantic information of items is a promising approach. Existing work integrating LLMs into recommendation tasks mainly follows two paradigms. The first paradigm uses LLMs directly for recommendations in natural language. These works design special prompts or use supervised fine-tuning to get the LLM to answer a given recommendation question. However, this paradigm can only determine the recommendation of one item at a time, and the frequency of LLM use increases linearly with the number of candidate items. Therefore, these methods are often only used in the re-ranking stage with only a few dozen candidate items. The second paradigm uses LLMs as encoders to provide item / user embeddings for similarity comparison and next item prediction. Figure 1 As shown, given a user interaction history represented in natural language, these works use an LLM to encode each token in the input text and then perform various pooling strategies on the output token embeddings to derive user embeddings. While promising, these works typically require training a large number of parameters, consuming significant computational resources. Furthermore, they struggle to effectively integrate ID-based collaborative information into the LLM, impacting recommendation effectiveness. Although efforts have been made to map collaborative embeddings to the LLM's linguistic space using simple linear projections, these methods fail to account for the diverse characteristics of different user types, potentially leading to poor recommendation results. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a parameter-efficient method for bidirectional fine-tuning of a large recommendation model that integrates collaborative information.

[0006] The technical solution of the present invention is as follows:

[0007] A parameter-efficient bidirectional fine-tuning method for a large recommendation model that integrates collaborative information, the method comprising:

[0008] Obtain the input text and normalize it;

[0009] Design a MoE architecture that includes multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text;

[0010] The parameters in the normalized text are trained, and the item with the highest score is selected as the next item to recommend to the user.

[0011] Furthermore, the normalized text includes a prefix and a suffix; the prefix is ​​used to capture collaborative information and adapt the large model to the recommendation task; the suffix is ​​used to capture information from the entire input text and convert it into the embedding representation required by the recommendation system.

[0012] Furthermore, a MoE architecture with multiple query experts is designed to handle different types of users, and user-specific collaboration information is integrated into the normalized text. Specifically:

[0013] The design incorporates a MoE architecture with multiple query experts, each responsible for handling a specific type of collaborative information and interacting with it through a Transformer block.

[0014] The router calculates scores for different query experts based on the user's interaction history and selects the expert with the highest score to process the current user.

[0015] The selected query expert interacts with collaborative information through the Transformer block.

[0016] Furthermore, the expert's score is obtained using the following formula:

[0017]

[0018] Where, r j (u) represents the score given by expert j to user u, p i,j (u) represents user u's preference or relevance to expert j in the i-th interaction, and N is the total number of user interactions.

[0019] Furthermore, the interaction with collaborative information through the Transformer block specifically involves interacting with the internal state of the expert network through self-attention and cross-attention mechanisms.

[0020] Furthermore, it also includes training through a loss function, specifically as follows:

[0021]

[0022] Where λ is a hyperparameter. For the loss of the main mission, Loss of auxiliary tasks.

[0023] Furthermore, the parameters in the trained normalized text are specifically as follows:

[0024] Freeze the parameters of the large recommendation model, calculate the similarity between the user embedding and the item embedding obtained from the large recommendation model, and select the item with the highest score as the next item to recommend to the user.

[0025] Furthermore, the computation of user embedding and item embedding specifically includes:

[0026]

[0027] Where s(u,i)∈R represents the probability that item i will become the next item browsed by user u; The dot product of user embedding and item embedding is ||h u ||and||h i || represent the Euclidean norms of user embedding and project embedding, respectively;

[0028] To predict the next item, iterate through each item i in the item set and select the item i^ with the highest score as the next item:

[0029]

[0030] Where I represents the item set, and argmax represents selecting item i that maximizes the similarity score s(u,i).

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] This invention adapts LLM to sequence recommendation through trainable prefixes and suffixes. Prefixes adapt LLM to recommendation tasks with collaborative information, while suffixes transform the LLM output from the linguistic space to the recommendation space, obtaining high-quality user / item embeddings. To effectively integrate ID-based collaborative information for more accurate recommendations, M-Former, a query Transformer based on a lightweight MoE, is introduced, using a set of query experts to capture diverse collaborative features of different user types. Finally, a multi-task loss function and a two-stage training strategy are employed to train the Laser for sequence recommendation. Extensive experiments on real-world datasets demonstrate that this invention can adapt LLM to an effective recommendation system in a parametrically efficient manner, significantly outperforming state-of-the-art methods. Attached Figure Description

[0033] The accompanying drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the specification and claims, to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0034] Figure 1 A schematic diagram of the method framework of the present invention is shown;

[0035] Figure 2 A schematic diagram showing the performance comparison of the Scientific dataset of the present invention under zero-sample and low-resource settings is illustrated. Detailed Implementation

[0036] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0037] This invention provides a method for bidirectional fine-tuning of a large recommendation model that integrates collaborative information with high parameter efficiency.

[0038] First, describe how to store the user interaction history Du = {Di1, Di2, ..., Di...} N Organize them into a coherent text Tu = {t1, t2, ..., t} W The proposed parameter is used as input for sequence recommendation in LLM. Then, we introduce how to adapt LLM to the recommendation task through the proposed parameter-efficient bidirectional fine-tuning method.

[0039] The user interaction history (Du) is organized into an LLM input text (Tu) for recommendation using a unified template. For example, given a user who has viewed "Kaytee Aspen Bedding Bag", "Guitar A-Frame Supports", and "KONG Wubba DogToy", this invention can normalize the corresponding input text (Tu) into the following form.

[0040] Please summarize the user's characteristics into a token based on their browsing history, arranged chronologically. The items the user has viewed are as follows:

[0041] >>1.Kaytee Aspen Bedding Bag (Brand: Kaytee, Category: Kaytee)

[0042] >>2. Guitar A-Frame Support (Brand: Sageworks, Category: Sageworks) ...

[0044] >>4. KONG Wubba Dog Toy (Brand: KONG, Category: KONG)

[0045] Using the designed template, the LLM can summarize a user's browsing history into a token (i.e., a suffix appended to the end of the input text) based on instructions in the input text. The corresponding output embedding can be used as the user embedding *hu*, for similarity comparison with item embeddings and next item recommendation. To obtain item embeddings, this invention also uses the same template. Specifically, for a specific item *i*, this invention treats it as a special user interaction history containing only that item. Therefore, this invention uses the same template to formulate the input text for the LLM and uses the output appended suffix embedding as the item embedding *hi*. In this way, this invention can obtain the embedding for each item *i* in item set *I*, and the original user-item similarity comparison for recommendation can be considered a special user-user similarity comparison. The advantage of this is that a unified template minimizes the impact of hard templates on LLM performance.

[0046] Given input text Tu = {t1, t2, ..., t} W}, it will be expanded into trainable prefixes and suffixes:

[0047]

[0048] Where P = {p1, p2, ..., p} L} refers to the prefix containing L preceding virtual tokens, and s refers to the suffix consisting of an additional virtual token. During model training, this invention freezes the parameters of the LLM and customizes them for the recommendation task by optimizing the trainable virtual tokens added to the prefix P and suffix s, which greatly reduces the size of the parameters that need to be trained.

[0049] The prefix P, containing L virtual tokens, is responsible for adapting the LLM to the recommended task through collaborative information. These virtual tokens can serve as placeholders, allowing the LLM to capture task-specific information during fine-tuning.

[0050] In addition to the prefix P, this invention appends a special trainable virtual token s, called a suffix, to the end of the input text Tu. Previous works attempted to obtain the user embedding by average pooling the token embeddings of the LLM output. However, most generative LLMs are based on masked attention mechanisms, meaning that only the last token is observable of the entire input. Therefore, these works may introduce noise by average pooling all output embeddings. In this work, the invention uses the appended trainable virtual token s to capture information about the entire input Tu, whose output embedding hs can be used as the user embedding hu for similarity comparison and next item prediction. Formally, the encoding process of the LLM can be represented as:

[0051]

[0052] Where e∈Rd represents the input token embedding of input Tu={p1,...,t1,...,s}, h∈Rd represents the corresponding output embedding, and d represents the hidden size of the LLM. Through the trainable suffix s, this invention can effectively transform the output embedding of the LLM from the language space to the recommendation space. When user interaction history or a single item is used as input to the LLM, this invention can directly use the output suffix embedding hs as the user embedding hu or the item embedding hi for further recommendation.

[0053] Given user embeddings hu∈Rd and item embeddings hi∈Rd obtained from LLM, this invention can calculate their similarity as follows:

[0054]

[0055] Where s(u,i)∈R represents the probability that item i will be the next item browsed by user u. To predict the next item, this invention iterates through each item i in the item set I and selects the item i^ with the highest score as the next item:

[0056]

[0057] This invention uses trainable prefixes P and suffixes s to adapt LLMs for recommendation. To achieve better recommendation results, this invention incorporates collaborative information through prefix P. Existing work attempts to project the collaborative embeddings encoded by ID-based sequence recommender systems into the language space of LLMs using a unified linear layer. However, this approach is too simplistic and fails to detect the diverse characteristics of different user types. To address this challenge, this invention introduces M-Former, a MoE-based query Transformer that uses a set of query experts to handle different user types and integrates user-specific collaborative information into prefix P.

[0058] like Figure 2 As shown, there are K query experts handling different types of users, and each expert contains L trainable virtual tokens. To select the most suitable expert to handle user-specific collaborative information, this invention sets up a router to calculate the scores of different experts for a given user. Formally, given user u, the user interaction history Su = {i1, i2, ..., i...} N The corresponding item ID sequence is IDu = {idi1, idi2, ..., idi}. N The input to the pre-trained ID-based sequence recommender system (frozen) is encoded as Cu∈RN×di, where N represents the length of the interaction history Su and di represents the hidden size of the sequence recommender system. Then, Cu is fed into a router, which in this work is implemented by a fully connected layer. Based on the N user interaction items embedded as Cu, the matching degree of the K query experts can be calculated as follows:

[0059]

[0060] Where Wr∈RK×di are the linear weights of the router, and m(u)∈RN×K. Then, the score of the i-th item to the j-th expert can be calculated as:

[0061]

[0062] And the final score of the j-th query expert can be obtained through:

[0063]

[0064] get.

[0065] Finally, the query expert with the highest score will be selected to handle the specific user u.

[0066] like Figure 1As shown, the selected query expert, containing L virtual tokens, is sequentially fed into Z Transformer blocks to interact with the collaborative information Cu. In this way, the query expert integrates the collaborative information into its L trainable virtual tokens, which are further used as the aforementioned prefix P = {p1, p2, ..., pL} to adapt the LLM to sequence recommendations while enhancing the collaborative information.

[0067] Formally, the query expert can be represented as E∈RL×dm, where dm is the hidden size of the M-Former. In a Transformer block, E is first encoded through a self-attention layer and then projected onto a query matrix Q, which interacts with a key / value matrix (K / V) projected from the ID-based collaborative embedding Cu in a cross-attention layer. In this way, the present invention updates the embedding E of the query expert and integrates ID-based collaborative information therein. Then, through a linear projection layer set on top of Z Transformer blocks, the query expert is projected onto the hidden size d of the LLM and used as a prefix P = {p1, p2, ..., p...} L This allows LLM to adapt to sequence recommendations.

[0068] A multi-task training strategy is proposed to jointly train the proposed Laser for LLM-based sequence recommendation.

[0069] The first training task is the item-to-item contrast (IIC) task, which is widely used for next item prediction. Following previous work, this invention uses the true next item as a positive example and all other items in item set I as negative examples. Formally, the item-to-item contrast loss is calculated as follows:

[0070]

[0071] The second training task is a load balancing task, designed to encourage load balancing among different query experts in the M-Former. As demonstrated in previous work, this task forces the router to assign users with different collaboration characteristics to different query experts, allowing each expert to be trained to achieve optimal collaboration information integration for its user group. Formally, the load balancing loss is calculated as follows:

[0072]

[0073] Where K is the number of experts queried, f j The proportion of projects allocated to the j-th expert can be calculated as follows:

[0074]

[0075] Where N is the number of items in the user interaction history. P in the formula...j The probability proportion assigned by the router to the j-th expert can be calculated as follows:

[0076]

[0077] In summary, the loss function used in this work is:

[0078]

[0079] Where λ is a hyperparameter that controls the weights of different tasks.

[0080] Recommendations are based on user-item embedding similarity comparisons. Since item embeddings are determined by corresponding trainable suffixes, they change after different training cycles. This invention employs a two-stage training strategy: first, finding the most suitable parameter weights to obtain high-quality item embeddings; then, further training the Laser based on these fixed item embeddings to achieve optimal recommendation results.

[0081] Specifically, in the first training phase, at the beginning of each period, the item embeddings are updated using the current weights A. Laser is then trained on I and validated at the end of each epoch based on the updated parameter weights A'. At the end of the first training phase, the embedding of the epoch with the best performance is selected. and corresponding parameter weights Used for the second training phase. In the second training phase, Laser uses... Initialize, then train for multiple epochs to further adapt to the fixed embedding. Finally, the parameter weights that produce the optimal verification performance are determined. Reserved, in and The test results represent Laser's final performance.

[0082] To evaluate the effectiveness of this invention, experiments were conducted on three categories of Amazon review datasets: "Industrial & Scientific," "Arts, Crafts & Sewing," and "Pet Supplies." Following previous work, this invention used a five-core dataset provided by the data source and filtered out items with missing titles. Then, this invention collected interactions from different users and sorted the interacting items in ascending order of timestamp. Statistical information for the preprocessed dataset is shown in Table 1. For modeling the semantic information of items, this invention selected item attributes including title, category, and brand.

[0083] Table 1 shows the statistics of the preprocessed dataset.

[0084]

[0085] Next, this invention first describes the MoE strategy, namely, how to select the most suitable query expert from a set of experts based on the collaborative characteristics of a specific user. Then, this invention explains how the selected query expert interacts with collaborative information encoded by a frozen ID-based sequence recommender system in the query Transformer.

[0086] This invention compares itself with many state-of-the-art baselines, including six traditional methods (SASRec, BERT4Rec, RecGURU, FDSA, ZESRec, RECFORMER) and three LLM-based methods (LLM4REC, ZESRec, LlamaRec). Furthermore, the frozen ID-based sequence recommender used in this invention is a pre-trained BERT4Rec, and the frozen LLM used is ChatGLM2-6B. All other trainable modules are randomly initialized.

[0087] Following previous work, this invention uses three popular metrics: Recall@N, NDCG@N, and MRR, where N is set to 10. For data partitioning, this invention employs leave-one-out, where the most recent item in the interaction history is used for testing, the second most recent item for validation, and the remaining items for training. This invention treats all items in the item set as candidate items and reports the average result on the test data.

[0088] Table 2 Performance Comparison of Different Methods

[0089]

[0090] Table 3 Comparison of parameter scales for different LLM tuning methods

[0091]

[0092] The results were compared with nine state-of-the-art baselines on three Amazon datasets. From the experimental results, the present invention yielded the following observations.

[0093] First, compared with other excellent sequence recommendation methods, this invention achieves significant improvements across all metrics on all datasets. For example, on the Pet dataset, compared with the second-best method, this invention improves Recall@10, NDCG@10, and MRR by approximately 11.37%, 13.27%, and 10.63%, respectively. This demonstrates that the framework proposed in this invention can successfully adapt LLM into an effective sequence recommender system. This invention benefits from the Bi-Tuning method, which effectively adapts LLM to sequence recommendation with collaborative information. Furthermore, when integrating collaborative information, the designed M-Former (a query Transformer based on MoE) captures diverse characteristics of different types of users for more accurate recommendations.

[0094] Furthermore, Table 3 compares the parameter sizes of different LLM tuning methods. It can be observed that the proposed Bi-Tuning method is more parameter-efficient, containing only 0.135M trainable parameters. The results show that Laser can significantly reduce the size of trainable parameters and achieve effective LLM adaptation through the proposed Bi-Tuning method, outperforming the state-of-the-art (SOTA) baseline. Note that Table 3 only lists the trainable parameter sizes of different LLM tuning methods. Laser's total number of trainable parameters is approximately 183.3M, which is also significantly less than 3% of the number of parameters in the LLM backbone ChatGLM2-6B.

[0095] To further demonstrate the effectiveness of this invention, experiments were conducted to examine its performance in zero-sample and low-resource scenarios. Specifically, this invention (using both item semantic information and ID-based collaboration information) was compared with two other types of baselines: BERT4Rec using only ID-based collaboration information and RECFORMER using only item semantic information. This invention first pre-trained these methods (except for ID-based BERT4Rec) on the Pet dataset and then tested their performance on another domain with no / limited training data.

[0096] from Figure 2In this invention, we can observe that: (1) The Laser performs best in zero-shot scenarios. Compared with other baselines, the Laser achieves significantly better performance (Recall@10 reaches 0.97, NDCG@10 reaches 0.58), even though it has not seen any items on the Scientific dataset. This invention attributes this superior performance to the design of the Bi-Tuning framework of this invention, which fully utilizes the generalization ability of LLM and effectively adapts LLM to sequence recommendation. (2) This invention only requires 5% of the training data to outperform other baselines that use 100% of the training data. Compared with the other two baselines, as the proportion of training data increases to 5%, the performance of the Laser can quickly rise to a very considerable level. This means that this invention only requires a very small amount of training data and training time to transfer the Laser trained on one domain to another unseen domain, with better results than other baselines that require more training data. This shows that the framework proposed in this invention can effectively transform LLM into a generalizable sequence recommender system.

[0097] To demonstrate the effectiveness of each module in this invention, an ablation study was conducted, and the results are presented in Table 4. This invention observes that: (1) the experimental results on both datasets are consistent. Removing any module leads to a significant decrease in Laser performance. (2) Without MoE, Recall@10, NDCG@10, and MRR decrease by an average of 8.28%, 8.63%, and 10.92%, respectively, indicating that introducing MoE helps the framework of this invention handle diverse collaborative features of different types of users, resulting in higher-quality recommendation results. Furthermore, without M-Former, these three metrics decrease by an average of 9.15%, 13.35%, and 16.51%, respectively. This demonstrates the importance of using ID-based collaborative information for more accurate recommendations, and that the M-Former of this invention effectively integrates collaborative information into LLM. (3) Without the prefix, Recall@10, NDCG@10, and MRR decrease significantly by an average of 20.89%, 24.97%, and 25.39%, respectively, indicating the important role of the prefix in adapting LLM to recommendation tasks. (4) Removing any training phase reduces the effectiveness of the Laser. Specifically, without the first training phase, Recall@10, NDCG@10, and MRR decrease by an average of 49.24%, 47.29%, and 48.28%, respectively, indicating the need to find appropriate parameter weights to obtain high-quality item embeddings. Furthermore, without the second training phase, the metrics also decrease by 9.13%, 7.28%, and 9.96%, respectively. This suggests that after obtaining suitable item embeddings, further training is needed to better adapt the Laser to fixed item embeddings and achieve optimal recommendation results.

[0098] Table 4 shows the results of the ablation study.

[0099]

[0100] This invention adapts LLM to sequence recommendation through trainable prefixes and suffixes. Prefixes adapt LLM to recommendation tasks with collaborative information, while suffixes transform the LLM output from the linguistic space to the recommendation space, obtaining high-quality user / item embeddings. To effectively integrate ID-based collaborative information for more accurate recommendations, M-Former, a query Transformer based on a lightweight MoE, is introduced, using a set of query experts to capture diverse collaborative features of different user types. Finally, a multi-task loss function and a two-stage training strategy are employed to train Laser for sequence recommendation. Extensive experiments on real-world datasets demonstrate that Laser can adapt LLM to an effective recommendation system in a parameter-efficient manner, significantly outperforming state-of-the-art methods.

[0101] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A parameter-efficient, fusion-cooperative-information, recommendation large model bidirectional fine-tuning method, characterized in that, The method includes: Obtain the input text and normalize it; Design a MoE architecture that includes multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text; The parameters in the normalized text are trained, and the item with the highest score is selected as the next item to recommend to the user. The normalized text includes a prefix and a suffix; the prefix is ​​used to capture collaborative information and adapt the large model to the recommendation task; the suffix is ​​used to capture information from the entire input text and convert it into the embedding representation required by the recommendation system. Design a MoE architecture that incorporates multiple query experts to handle different types of users and integrate user-specific collaboration information into the normalized text. Specifically: The design incorporates a MoE architecture with multiple query experts, each responsible for processing a specific type of collaborative information and interacting with it through a Transformer block. The router calculates scores for different query experts based on the user's interaction history and selects the expert with the highest score to process the current user. The selected query expert interacts with collaborative information through the Transformer block; The expert's score is obtained using the following formula: in, r j ( u (This indicates an expert) j For users u The score, p i,j ( u ) represents the user u In the i In the interaction for experts j preferences or relevance, N It represents the total number of user interactions; The parameters in the normalized text after training are as follows: Freeze the parameters of the recommendation model, calculate the similarity between the user embedding and the item embedding obtained from the recommendation model, and select the item with the highest score as the next item to recommend to the user. The computing the user embedding and the item embedding specifically are: wherein, representing items will be a user probability of next browsing item; representing the dot product of the user embedding and the item embedding, and representing the Euclidean norm of the user embedding and the item embedding, respectively To predict the next item, iterate through each item in the set of items and select the item with the highest score as the next item: wherein, I represents a set of items, argmax represents selecting the item that maximizes the similarity score s u , i ) the maximum i .​ 2. The method for bidirectional fine-tuning of a large recommendation model with efficient parameter fusion and collaborative information as described in claim 1, characterized in that, The interaction with collaborative information via Transformer blocks specifically involves interacting with the internal state of the expert network through self-attention and cross-attention mechanisms.

3. The parameter-efficient few-shot learning method of claim 1, wherein, This also includes training via a loss function, specifically: wherein, λ is a hyperparameter, is a loss for the main task, is a loss for the auxiliary task.

Images