Large model article recommendation method based on low-rank expert fusion

Abstract

The invention discloses a large-model article recommendation method based on low-rank expert fusion. The method comprises the following steps: 1, constructing a text data set of a source domain and a target domain by utilizing a public recommendation data set; 2, constructing a large model, a source domain low-rank expert model and a target domain low-rank expert model, and performing fine tuning on a source domain text data set and a target domain text data set to respectively obtain corresponding expert model parameters; 3, guiding learning of fusion weight through Shannon entropy loss, and performing expert model parameter fusion; and 4, in a model reasoning stage, predicting a next to-be-interacted article of the user by utilizing the fused low-rank expert and the large model. According to the method, the understanding ability of the large model to the recommended knowledge can be improved, the large model can quickly adapt to the recommended knowledge in the new field, and accurate recommendation is brought to the user.

Description

Technical Field

[0001] The present invention relates to the field of large model item recommendation, and specifically to a new large model item recommendation method based on low-rank expert fusion. Background Art

[0002] Large-scale model item recommendation can understand users' various needs through dialogue, recommend items to users based on their preferences, make the recommendation results more personalized, and support the processing of multimodal data; large models have extensive general world knowledge, and can provide users with detailed descriptions of items and detailed reasons for recommendations while recommending, and can quickly respond to changes in user behavior; in the face of cold start problems in recommendations, a small amount of data can be used to quickly adapt to new users or new items; these advantages have made large-scale model recommendation a huge success. In order to enable large models to fully understand recommendation knowledge, researchers have proposed many methods, which can be summarized into two paradigms.

[0003] The first paradigm is the breadth-oriented paradigm. This type of research typically integrates recommendation data from multiple domains or tasks to build broad and general recommendation knowledge, aiming to develop general recommendation systems based on large models. The core of this paradigm is to integrate multi-source data to construct instruction fine-tuning datasets and design instruction templates applicable to various tasks for large-scale model training.

[0004] The second paradigm is the depth-oriented paradigm. This type of research aims to enable large models to deeply understand recommendation tasks within specific domains. Its focus includes deeply extracting domain-specific recommendation knowledge, such as collaborative filtering information, and developing efficient and effective methods to align large models with recommendation tasks. Specifically, compared to the huge number of parameters in large models, downstream tasks usually do not have enough data to fine-tune all parameters. Therefore, parameter-efficient fine-tuning methods become the best choice for training large models. Among them, lightweight low-rank adapters (LoRA) are a representative work. Leveraging or enhancing LoRA fine-tuning techniques and designing data-efficient fine-tuning strategies are also one of the research focuses.

[0005] However, both existing paradigms have their own strengths and weaknesses. In the breadth-oriented paradigm, large models master general recommendation knowledge and generalize well to various recommendation scenarios, but may perform poorly in specific domains. The depth-oriented paradigm learns users' unique preferences and is suitable for hot-start recommendations in specific domains, but its performance in specific domains is difficult to directly transfer to other domains. This is due to the distribution shift between training and test data, which poses significant challenges, especially when new users or new items appear, or when training data is sparse. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the present invention proposes a large-model item recommendation method based on low-rank expert fusion, in order to enable the large model to improve its ability to understand the recommendation knowledge, and enable the large model to quickly adapt to the recommendation knowledge in new fields, providing accurate recommendations for users.

[0007] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0008] The large-model item recommendation method based on low-rank expert fusion of the present invention is characterized in that it includes the following steps:

[0009] Step 1: Build a source domain text dataset and target domain text dataset :

[0010] Step 2: Build large models, source domain low-rank expert models and target domain low-rank expert models with the same structure, and and After fine-tuning the expert model with the same input structure, the optimal parameters of the expert model are obtained. ;

[0011] Step 3: Construct the L-layer fusion low-rank expert model, The source domain low-rank expert model of the layer and The target domain low-rank expert model of the L layer is composed of the target domain low-rank expert model, and the fusion weight is guided by the Shannon entropy loss, so as to fine-tune the fusion weight of the fusion low-rank expert model of the L layer to obtain the optimal source domain weight and the optimal target domain weight ,in, Indicates the The optimal source domain weight of the layer, Indicates the The optimal target domain weight of the layer:

[0012] Step 4: In the inference phase, use the combined parameters The prediction model predicts the items that users will interact with.

[0013] The large-model item recommendation method based on low-rank expert fusion described in the present invention is also characterized in that step 1 includes the following steps:

[0014] Step 1.1: Obtain source domain dataset and target domain dataset ,in, Indicates the Source domain interaction records, Indicates the Source domain users in the source domain interaction records, Indicates the Source domain items in source domain interaction records, Indicates the The source domain timestamp in the source domain interaction record, Indicates the total number of source domain interaction records, Indicates the Entry domain interaction record, Indicates the The target domain user in the target domain interaction record. Indicates the The target domain item in the target domain interaction record, express The target domain timestamp in the target domain interaction record of the entry; represents the total number of target domain-target domain interaction records;

[0015] Step 1.2: Source domain dataset The source domain items interacted by the same source domain user in the source domain are arranged in order according to the timestamp, and the item title sequence set of all source domain users is obtained. ;

[0016] Step 1.3: Target domain dataset The target domain items interacted by the same target domain user in the target domain are arranged in order according to the timestamp, and the item title sequence set of all target domain users is obtained. ;

[0017] Step 1.4, according to and Construct source domain text datasets separately and target domain text dataset ,in, Indicates the source domain text data, Indicates the Source domain text, express Tags, Indicates the total number of source domain data, Indicates the Item domain data, Indicates the Entry field text, express Tags, Indicates the total number of target domain data.

[0018] Furthermore, the step 2 includes the following steps:

[0019] Step 2.1, build a large model, including: embedding layer, The transformer submodule of the layer defines For the The pre-trained weight matrix of the transformer submodule of the layer, represents the length of the matrix, Indicates the width of the matrix;

[0020] Step 2.2, let The first layer of the source domain low-rank expert model The two source domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ,make The first layer of the low-rank expert model in the target domain The two target domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ; represents the rank of the low-rank matrix;

[0021] Step 2.3, Input into the large model and after being processed by the embedding layer, the initial Source domain embedding representation ;

[0022] Step 2.4 After inputting the L-layer transformer submodule and the L-layer source domain low-rank expert model for fusion processing, the formula (1) is used to obtain the first Layer Source domain embedding representation , thus obtaining the Lth layer Source domain embedding representation :

[0023] (1)

[0024] In formula (1), Indicates the Layer Strip source domain embedding representation;

[0025] Step 2.5: Use formula (2) to construct the embedding layer, the transformer submodule of the L layer, and the fine-tuning optimization target of the source domain low-rank expert model of the L layer. , and fine-tune the source domain low-rank expert model of the L layer to obtain the optimal parameters of the fine-tuned source domain low-rank expert model ,in, and Indicates the The two optimal source domain low-rank matrices of the layer:

[0026] (2)

[0027] In formula (2), is the parameter of the transformer submodule of the L layer, = is the parameter of the source domain low-rank expert model of the L layer, express Prediction probability;

[0028] Step 2.6, Input large model and The optimal parameters of the target domain low-rank expert model after fine-tuning are obtained according to the process of steps 2.3-2.5. ,in, and Indicates the The two optimal target domain low-rank matrices of the layer.

[0029] Furthermore, the step 3 includes the following steps:

[0030] Step 3.1: Use formula (3) and formula (4) to obtain The two fused low-rank matrices of the fused low-rank expert model of the layer and , thus obtaining The fusion parameters of the fusion low-rank expert model of the layer ;

[0031] (3)

[0032] (4)

[0033] In formula (4) and formula (5), Indicates the The fusion weight of the source domain low-rank expert model of the layer, Indicates the The fusion weight of the target domain low-rank expert model of the layer;

[0034] Step 3.2: Follow the process of step 2.3 to Input into the large model for processing to obtain the initial Strip fusion embedding representation ;

[0035] Step 3.3 After inputting into the transformer submodule of L layer and the fusion low-rank expert model of L layer for fusion processing, the formula (5) is used to obtain the first Layer Strip fusion embedding representation , thus obtaining the Lth layer Strip fusion embedding representation :

[0036] (5)

[0037] In formula (5), Indicates the Layer Strip fusion embedding representation;

[0038] Step 3.4: Use formula (6) to construct the fine-tuning optimization target of the embedding layer, the transformer submodule of the L layer, and the fusion low-rank expert model of the L layer. , and The fusion weight of the source domain low-rank expert model of the layer and The fusion weight of the target domain low-rank expert model of the layer Fine-tune to obtain the optimal source domain fusion weight and the optimal target domain fusion weight :

[0039] (6)

[0040] In formula (6), yes Predicted to be the The probability of the categories, Indicates the total number of categories.

[0041] Furthermore, step 4 includes the following steps:

[0042] Step 4.1: Build a prediction model and use equations (7), (8) and (9) to get the combined parameters of the prediction model: :

[0043] (7)

[0044] (8)

[0045] (9)

[0046] Step 4.2: Using the merge parameter The prediction model predicts the question text input by the user and obtains the item sequence of the user interaction.

[0047] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the large model item recommendation method, and the processor is configured to execute the program stored in the memory.

[0048] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the large model item recommendation method when the computer program is executed by a processor.

[0049] Compared with the prior art, the beneficial effects of the present invention are embodied in:

[0050] 1. The present invention proposes a new method to fuse two paradigms, namely low-rank expert fusion, which fuses the large model with low-rank experts in the source domain who have general domain recommendation knowledge and low-rank experts in the target domain who have specific domain knowledge. This allows the large model to have the advantages of both experts and make up for their disadvantages. The fusion process is simple and efficient, which improves the large model's mastery of recommendation knowledge and the accuracy of recommendations.

[0051] 2. The present invention can generalize the large recommendation model to various recommendation scenarios. For example, when faced with a new recommendation field, it only needs to train a new low-rank expert in the target domain to achieve rapid generalization. Even in extreme cases, when there is no training data in the new field, the present invention can still use the low-rank expert in the source domain to work. For new user or new item recommendation scenarios, the present invention can dynamically balance generalization capabilities and domain-specific capabilities to provide accurate recommendations. In addition, with the contextual learning capabilities of the large model, the present invention naturally has task generalization capabilities, for example, it can generate explainable recommendation results.

[0052] 3. The present invention is designed to be easy to use and supports plug-and-play integration. The source domain low-rank experts only need to be trained once, and the target domain low-rank experts can be fused after minimal fine-tuning. The fusion method does not add additional parameters or change the model structure, making it possible to combine multiple existing open source technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flowchart of the large-model item recommendation method based on low-rank expert fusion of the present invention. DETAILED DESCRIPTION

[0054] In this embodiment, a large-model item recommendation method based on low-rank expert fusion is designed to integrate the advantages of the two paradigms and improve the generalization ability and domain-specific ability of the large model. Specifically, Figure 1 As shown, the method is carried out in the following steps:

[0055] Step 1: Build a source domain text dataset and target domain text dataset :

[0056] Step 1.1: Obtain source domain dataset and target domain dataset ,in, Indicates the Source domain interaction records, Indicates the Source domain users in the source domain interaction records, Indicates the Source domain items in source domain interaction records, Indicates the The source domain timestamp in the source domain interaction record, Indicates the total number of source domain interaction records, Indicates the Entry domain interaction record, Indicates the The target domain user in the target domain interaction record. Indicates the The target domain item in the target domain interaction record, express The target domain timestamp in the target domain interaction record of the entry; represents the total number of target domain-target domain interaction records;

[0057] In this specific example, the present invention uses the publicly available recommendation datasets Amazon Review and MovieLens. The specific data is shown in Tables 1 and 2, where density represents the density of interactions within the dataset, calculated as the number of interactions divided by (number of users × number of items). Both datasets contain records of user-interaction items, each containing a user ID, item ID, and interaction timestamp. The datasets also contain item information, including item ID, title, category, and description. The Amazon Review dataset was cleaned using a common five-core filtering technique, removing users and items with fewer than five interaction records. These items were then used as cold-start target items to filter out the cold-start test set from the user interaction records. Due to the density of the MovieLens dataset, items with fewer than 20 interaction records were removed and used as cold-start target items. For the Amazon Review dataset, this paper uses eleven domains, namely Clothing (clothing, shoes and jewelry), Cell (mobile phones and accessories), Grocery (groceries and food), Health (health and personal care), Home (home and kitchen), Pet (pet supplies), Tools (tools and home decoration), Videos (video games), Beauty (beauty), Toys (toys) and Sports (sports and outdoor); for the MovieLens dataset, this paper uses two domains, 1M (one million interaction data) and 10M (ten million interaction data), and there is no overlap in interaction records in different domains. For each domain in the Amazon Review dataset, this paper mixes the data of the eight domains of Clothing, Cell, Grocery, Health, Home, Pet, Tools and Videos as the source domain dataset, and uses the data of the three domains of Beauty, Toys and Sports as the target domain dataset; for the MovieLens dataset, the Movielens-10M data is used as the source domain dataset, and the Movielens-1M data is used as the target domain dataset.

[0058] surface Detailed data of each field in the Amazon Review dataset

[0059]

[0060] Table 2 Detailed data of each field in the MovieLens dataset

[0061]

[0062] Step 1.2: Source domain dataset The source domain items interacted by the same source domain user in the source domain are arranged in order according to the timestamp, and the item title sequence set of all source domain users is obtained. ;

[0063] Step 1.3: Target domain dataset The target domain items interacted by the same target domain user in the target domain are arranged in order according to the timestamp, and the item title sequence set of all target domain users is obtained. .

[0064] Step 1.4, according to and Construct source domain text datasets separately and target domain text dataset ,in, Indicates the source domain text data, Indicates the Source domain text, express Tags, Indicates the total number of source domain data, Indicates the Item domain data, Indicates the Entry field text, express Tags, Indicates the total number of target domain data;

[0065] In the specific implementation, the input template of the text dataset is: the user has purchased [user interaction history]. Please recommend 3 items that the user may purchase next from the candidate set based on the user's purchase history, sorting them from high to low likelihood. The candidate set is [candidate items]. The text output template is the sorting of the three item titles. The higher the likelihood, the higher the title.

[0066] Step 2: Build a large model, a source domain low-rank expert model, and a target domain low-rank expert model with the same structure, and fine-tune the expert model with the same structure to obtain the optimal parameters of the expert model:

[0067] Step 2.1, build a large model, including: embedding layer, The transformer submodule of the layer defines For the The pre-trained weight matrix of the transformer submodule of the layer, represents the length of the matrix, Indicates the width of the matrix; in the specific implementation, the constructed large model is the Qwen2-7B-Instruct large model of Tongyi Qianwen, , and Determined by the specific loaded large model and its submodules.

[0068] Step 2.2, let The first layer of the source domain low-rank expert model The two source domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ,make The first layer of the low-rank expert model in the target domain The two target domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ; Indicates the rank of the low-rank matrix; in the specific implementation, the LoRA method is used to construct the low-rank expert. Set to 16.

[0069] Step 2.3, Input into the large model and after being processed by the embedding layer, the initial Source domain embedding representation ;

[0070] Step 2.4 After inputting the L-layer transformer submodule and the L-layer source domain low-rank expert model for fusion processing, the formula (1) is used to obtain the first Layer Source domain embedding representation , thus obtaining the Lth layer Source domain embedding representation :

[0071] (1)

[0072] In formula (1), Indicates the Layer Strip source domain embedding representation;

[0073] Step 2.5: Use formula (2) to construct the fine-tuning optimization target of the embedding layer, the transformer submodule of the L layer, and the source domain low-rank expert model of the L layer. , and fine-tune the source domain low-rank expert model of L layer, the specific implementation is as follows Figure 1 As shown in part (a), during the fine-tuning process, the parameters of the embedding layer and transformer submodule of the large model are frozen and do not participate in the parameter update. Only the parameters of the source domain low-rank expert are updated, thereby obtaining the optimal parameters of the fine-tuned source domain low-rank expert model. ,in, and Indicates the The two optimal source domain low-rank matrices of the layer:

[0074] (2)

[0075] In formula (2), are the parameters of the transformer submodule of the L layer, = is the parameter of the source domain low-rank expert model of the L layer, express Prediction probability.

[0076] Step 2.6, Input large model and The target domain low-rank expert model of the layer is fine-tuned according to the process of steps 2.3 to 2.5. The specific implementation is as follows Figure 1 As shown in part (b), similar to step 2.5, only the parameters of the low-rank expert in the target domain are updated, and the optimal parameters of the low-rank expert model in the target domain are obtained after fine-tuning. ,in, and Indicates the The two optimal target domain low-rank matrices of the layer.

[0077] Step 3: Construct the L-layer fusion low-rank expert model, The source domain low-rank expert model of the layer and The target domain low-rank expert model of the L layer is composed of the target domain low-rank expert model, and the fusion weight is guided by the Shannon entropy loss, so as to fine-tune the fusion weight of the fusion low-rank expert model of the L layer to obtain the optimal source domain weight and the optimal target domain weight ,in, Indicates the The optimal source domain weight of the layer, Indicates the The optimal target domain weight of the layer:

[0078] Step 3.1: Use formula (3) and formula (4) to obtain The two fused low-rank matrices of the fused low-rank expert model of the layer and , thus obtaining The fusion parameters of the fusion low-rank expert model of the layer ;

[0079] (3)

[0080] (4)

[0081] In formula (4) and formula (5), Indicates the The fusion weight of the source domain low-rank expert model of the layer, Indicates the The fusion weight of the target domain low-rank expert model of the layer;

[0082] Step 3.2: Follow the process of step 2.3 to Input into the large model for processing to obtain the initial Strip fusion embedding representation ;

[0083] Step 3.3 After inputting into the transformer submodule of L layer and the fusion low-rank expert model of L layer for fusion processing, the formula (5) is used to obtain the first Layer Strip fusion embedding representation , thus obtaining the Lth layer Strip fusion embedding representation :

[0084] (5)

[0085] In formula (5), Indicates the Layer Strip fusion embedding representation.

[0086] Step 3.4: Use formula (6) to construct the fine-tuning optimization target of the embedding layer, the transformer submodule of the L layer, and the fusion low-rank expert model of the L layer. , and The fusion weight of the source domain low-rank expert model of the layer and The fusion weight of the target domain low-rank expert model of the layer Fine-tune to obtain the optimal source domain fusion weight and the optimal target domain fusion weight :

[0087] (6)

[0088] In formula (6), yes Predicted to be the The probability of the categories, Indicates the total number of categories.

[0089] Specific implementation such as Figure 1As shown in part (c) of the figure, during the fine-tuning process, the output of the generative model is in the form of autoregressive generation. The generation of an embedding is affected by the above. The output of the present invention is derived from the item titles in the candidate set. When the first few embeddings of an item title are determined, the complete title of the item is highly likely to have been determined. Therefore, in actual operation, only the first few embeddings of each item title are involved in the Shannon entropy calculation. The number of embeddings involved in the calculation is determined by the length of the item titles in the dataset. In formula (6), In the specific implementation, the output embedding is predicted to be the probability of embedding in the entire embedding table, Represents the total number of embedded categories in the embedding table. Large models have a large number of parameters, so gradient checkpointing is used during fine-tuning to reduce memory consumption.

[0090] Step 4: Predict the items the user will interact with during the inference phase:

[0091] Step 4.1: Build a prediction model and use equations (7), (8) and (9) to get the combined parameters of the prediction model: :

[0092] (7)

[0093] (8)

[0094] (9)

[0095] Step 4.2: Using the merge parameter The prediction model predicts the question text input by the user and obtains the item sequence of the user interaction.

[0096] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0097] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0098] In practice, since the merged model structure remains the same as the original large model, VLLM technology is used to accelerate inference, significantly reducing the amount of video memory and time consumed by inference. During the model testing phase, a test text dataset is input, and the inference results of the large model are saved in output order. Each test data item is labeled, and the Normalized Discounted Cumulative Gain (NDCG) is calculated based on the position of the label in the output sequence to evaluate the quality of the large model's item recommendations. The test set is divided into hot start and cold start scenarios. The hot start scenario refers to the labeled item having appeared in the training set, while the cold start scenario refers to the labeled item not having appeared in the training set.The final experimental results are shown in Table 3 and Table 4. Among them, the compared methods are BPR-MF (BPR: Bayesian personalized ranking from implicit feedback), GRU4Rec (Session-based Recommendations with Recurrent Neural Networks), SASRec (Self-attentive sequential recommendation), FMLP-Rec (Filter-enhanced MLP is All YouNeed for Sequential Recommendation), UniSRec (Towards universal sequence representation learning) for recommender systems), VQ-Rec (Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders), Qwen2-7B (Qwen2 Technical Report), RecFormer (Text Is All You Need: LearningLanguage Representations for Sequential Recommendation), P5 (Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predictparadigm (p5)), TALLRec(Tallrec: An effective and efficient tuning framework to align large language model with recommendation.), MoLoRec-G means that the present invention only uses the source domain experts and the large model, MoLoRec-S means that the present invention only uses the target domain experts and the large model, and MoLoRec means using the complete steps of the present invention.

[0099] Table 3 Experimental results of hot start scenario

[0100]

[0101] Table 4 Cold start scenario experimental results

[0102]

[0103] In Tables 3 and 4, the underlined data are suboptimal results, and the bolded data are optimal results. It can be seen that the present invention has achieved good results in various fields in both hot start and cold start scenarios. These experimental results fully demonstrate the effectiveness of the method of the present invention. On the one hand, it strengthens the large model's understanding and mastery of specific domain knowledge, and on the other hand, it also improves the generalization and versatility of the large model.

Claims

1. A large-model item recommendation method based on low-rank expert fusion, characterized in that: The steps include: Step 1: Build a source domain text dataset and target domain text dataset : Step 2: Build large models, source domain low-rank expert models and target domain low-rank expert models with the same structure, and and After fine-tuning the expert model with the same input structure, the optimal parameters of the expert model are obtained. ; Step 3: Construct the L-layer fusion low-rank expert model, The source domain low-rank expert model of the layer and The target domain low-rank expert model of the L layer is composed of the target domain low-rank expert model, and the fusion weight is guided by the Shannon entropy loss, so as to fine-tune the fusion weight of the fusion low-rank expert model of the L layer to obtain the optimal source domain weight and the optimal target domain weight ,in, Indicates the The optimal source domain weight of the layer, Indicates the The optimal target domain weight of the layer: Step 4: In the inference phase, use the combined parameters The prediction model predicts the items that users will interact with.

2. The large-model item recommendation method based on low-rank expert fusion according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1: Obtain source domain dataset and target domain dataset ,in, Indicates the Source domain interaction records, Indicates the Source domain users in the source domain interaction records, Indicates the Source domain items in source domain interaction records, Indicates the The source domain timestamp in the source domain interaction record, Indicates the total number of source domain interaction records, Indicates the Entry domain interaction record, Indicates the The target domain user in the target domain interaction record. Indicates the The target domain item in the target domain interaction record, express The target domain timestamp in the target domain interaction record of the entry; represents the total number of target domain-target domain interaction records; Step 1.2: Source domain dataset The source domain items interacted by the same source domain user in the source domain are arranged in order according to the timestamp, and the item title sequence set of all source domain users is obtained. ; Step 1.3: Target domain dataset The target domain items interacted by the same target domain user in the target domain are arranged in order according to the timestamp, and the item title sequence set of all target domain users is obtained. ; Step 1.4, according to and Construct source domain text datasets separately and target domain text dataset ,in, Indicates the source domain text data, Indicates the Source domain text, express Tags, Indicates the total number of source domain data, Indicates the Item domain data, Indicates the Entry field text, express Tags, Indicates the total number of target domain data.

3. The large-model item recommendation method based on low-rank expert fusion according to claim 2, characterized in that: The step 2 comprises the following steps: Step 2.1, build a large model, including: embedding layer, The transformer submodule of the layer defines For the The pre-trained weight matrix of the transformer submodule of the layer, represents the length of the matrix, Indicates the width of the matrix; Step 2.2, let The first layer of the source domain low-rank expert model The two source domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ,make The first layer of the low-rank expert model in the target domain The two target domain low-rank matrices of the source domain low-rank expert model of the layer are recorded as and ; represents the rank of the low-rank matrix; Step 2.3, Input into the large model and after being processed by the embedding layer, the initial Source domain embedding representation ; Step 2.4 After inputting the L-layer transformer submodule and the L-layer source domain low-rank expert model for fusion processing, the formula (1) is used to obtain the first Layer Source domain embedding representation , thus obtaining the Lth layer Source domain embedding representation : (1) In formula (1), Indicates the Layer Strip source domain embedding representation; Step 2.5: Use formula (2) to construct the fine-tuning optimization target of the embedding layer, the transformer submodule of the L layer, and the source domain low-rank expert model of the L layer. , and fine-tune the source domain low-rank expert model of the L layer to obtain the optimal parameters of the fine-tuned source domain low-rank expert model ,in, and Indicates the The two optimal source domain low-rank matrices of the layer: (2) In formula (2), are the parameters of the transformer submodule of the L layer, = is the parameter of the source domain low-rank expert model of the L layer, express Prediction probability; Step 2.6, Input large model and The optimal parameters of the target domain low-rank expert model after fine-tuning are obtained according to the process of steps 2.3-2.

5. ,in, and Indicates the The two optimal target domain low-rank matrices of the layer.

4. The large-model item recommendation method based on low-rank expert fusion according to claim 3 is characterized in that: The step 3 comprises the following steps: Step 3.1: Use formula (3) and formula (4) to obtain The two fused low-rank matrices of the fused low-rank expert model of the layer and , thus obtaining The fusion parameters of the fusion low-rank expert model of the layer ; (3) (4) In formula (4) and formula (5), Indicates the The fusion weight of the source domain low-rank expert model of the layer, Indicates the The fusion weight of the target domain low-rank expert model of the layer; Step 3.2: Follow the process of step 2.3 to Input into the large model for processing to obtain the initial Strip fusion embedding representation ; Step 3.3 After inputting into the transformer submodule of L layer and the fusion low-rank expert model of L layer for fusion processing, the formula (5) is used to obtain the first Layer Strip fusion embedding representation , thus obtaining the Lth layer Strip fusion embedding representation : (5) In formula (5), Indicates the Layer Strip fusion embedding representation; Step 3.4: Use formula (6) to construct the fine-tuning optimization target of the embedding layer, the transformer submodule of the L layer, and the fusion low-rank expert model of the L layer. , and The fusion weight of the source domain low-rank expert model of the layer and The fusion weight of the target domain low-rank expert model of the layer Fine-tune to obtain the optimal source domain fusion weight and the optimal target domain fusion weight : (6) In formula (6), yes Predicted to be the The probability of the categories, Indicates the total number of categories.

5. The large-model item recommendation method based on low-rank expert fusion according to claim 4 is characterized in that: The step 4 comprises the following steps: Step 4.1: Build a prediction model and use equations (7), (8) and (9) to get the combined parameters of the prediction model: : (7) (8) (9) Step 4.2: Using the merge parameter The prediction model predicts the question text input by the user and obtains the item sequence of the user interaction.

6. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the large model item recommendation method according to any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large model item recommendation method according to any one of claims 1 to 5 are executed.

Images