Large model cross-domain sequence recommendation method based on low-rank module knitting fusion
By employing a low-rank module weaving fusion method, the problem of reduced model performance in cross-domain sequence recommendation of large models is solved. This method achieves high accuracy in the target domain and cross-domain knowledge transfer, generating interpretable recommendation results that are applicable to various cross-domain recommendation scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI UNIV OF TECH
- Filing Date
- 2026-01-06
- Publication Date
- 2026-05-12
AI Technical Summary
Existing cross-domain sequence recommendation methods for large models require retraining when faced with changes in the scenario, which leads to reduced model performance and easily causes cross-domain preferences and gradient conflicts, making it difficult to achieve effective knowledge transfer in the target domain.
A low-rank module weaving fusion method is adopted. By constructing a hybrid training dataset and fine-tuning the low-rank expert modules of the target domain and the source domain, and combining the fusion parameters with a weighted averaging strategy, a fused low-rank expert module is formed for sequence recommendation of users in the target domain.
It improves the recommendation accuracy and cross-domain knowledge transfer capability of large models in the target domain, effectively transfers user behavior knowledge in cross-domain and cross-platform scenarios, generates interpretable recommendation results, and is easy to integrate and use.
Smart Images
Figure CN122019869A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of large model item recommendation technology, specifically to a large model cross-domain sequence recommendation method based on low-rank module weaving fusion. Background Technology
[0002] Large-scale cross-domain sequence recommendation leverages textual instructions to understand user interaction logic across multiple source domains, breaking the reliance of traditional recommendations on overlapping cross-domain users / items. Simultaneously, it utilizes the general world knowledge of large-scale models to supplement semantic information of items, providing users with personalized and interpretable recommendation results. Large-scale models support efficient parameter fine-tuning (such as LoRA), adapting to multi-domain data without updating all parameters, balancing training costs and knowledge transfer efficiency. In scenarios with sparse target domain data or expansion into new domains, it can rapidly improve target domain recommendation performance through source domain knowledge. These advantages make it an important direction for solving the pain points of cross-domain recommendation. To enable large-scale models to more efficiently transfer multi-domain recommendation knowledge, existing methods can be summarized into a paradigm—the data merging paradigm. This paradigm typically integrates recommendation data from multiple domains and tasks, constructing a unified instruction fine-tuning dataset to train a single large-scale model to adapt to cross-domain recommendation tasks across all scenarios. The core of this paradigm lies in designing a general instruction template to align multi-domain interaction data, allowing the large-scale model to master multi-domain recommendation knowledge through a single training iteration, aiming to develop a general cross-domain recommendation system that "covers multiple scenarios with a single model."
[0003] However, we believe that this paradigm has two limitations: first, it is inflexible, as adding or removing a domain as needed for a scenario will cause the model to be retrained from scratch; second, it causes data conflicts, as forced multi-domain co-training can easily lead to cross-domain preference conflicts and gradient conflicts during the training process, resulting in a decrease in the model's recommendation performance in the target domain.
[0004] These limitations have prompted us to seek new paradigms for building better cross-domain sequence recommendation models.
[0005] Fortunately, model fusion offers a viable solution. Model fusion is a paradigm quite different from data merging; it combines the parameters of multiple single-task models, enabling a single model to perform multiple tasks effectively without retraining. This is a promising technique, and if successfully applied to the field of cross-domain sequence recommendation, it could naturally address the two limitations mentioned above.
[0006] The naive model fusion paradigm aims to achieve knowledge transfer by fusing parameters from independently trained models across different domains. Its key aspects include training a large model or a low-rank module of a large model tailored to each domain separately, and employing simple strategies such as weighted averaging and interpolation, or cutting-edge model fusion methods, to fuse the parameters of multiple single-domain models, avoiding the high cost of retraining on full datasets. Specifically, considering the large number of parameters in large models, this paradigm often uses lightweight low-rank adapters such as LoRA for domain model training, updating only a small number of low-rank matrix parameters to adapt to single-domain recommendation tasks, while using parameter fusion as the core means to achieve cross-domain knowledge transfer. However, in the naive model fusion paradigm, the source domain model is not adapted to the target domain distribution, and direct fusion can easily lead to model performance degradation. In scenarios with sparse target domain data or cross-platform environments, effective source domain knowledge transfer is difficult to achieve, and it may even impair the model's original recommendation performance in the target domain. Summary of the Invention
[0007] To overcome the limitations of existing large-scale model cross-domain sequence recommendation paradigms, this invention proposes a large-scale model cross-domain sequence recommendation method based on low-rank module weaving fusion, aiming to enhance the large-scale model's understanding of cross-domain recommendation knowledge and deliver more accurate recommendation performance to users in the target domain.
[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion includes: Obtain the target domain dataset and multiple source domain datasets, and construct corresponding text training datasets for each. Merge the text training datasets of each source domain with the text training dataset of the target domain to form a mixed training dataset for each source domain. Each interaction record in the target domain dataset and the source domain datasets includes the user, the item the user interacted with, and the interaction timestamp. Based on the target domain text training dataset, the target domain low-rank expert module loaded into the large model is fine-tuned to obtain the optimized target domain low-rank expert module. Based on each hybrid training dataset, the hybrid low-rank expert module loaded into the large model is fine-tuned to obtain the optimized hybrid low-rank expert module corresponding to each hybrid training dataset. The optimal parameters of the optimized target domain low-rank expert module and all optimized hybrid low-rank expert modules are weighted and fused to obtain the fused low-rank expert module. The low-rank expert module is loaded into the large model for inference and prediction of sequence recommendation tasks for target domain users.
[0009] In one embodiment, obtaining the target domain dataset and multiple source domain datasets includes: Obtain the target domain dataset and the nth source domain dataset , , The total number of data sets in the source domain; where, express The Source domain interaction records, Indicates the first The source domain user in the source domain interaction record, Indicates the first Source domain items in the source domain interaction log. Indicates the first The source domain timestamp in the source domain interaction record; Indicates the first The total number of source domain interaction records in each source domain dataset; Represents the first of the target domain datasets Target field interaction records, Indicates the first The target domain user in the target domain interaction record, Indicates the first Target domain items in the target domain interaction record. express The target domain timestamp in the target domain interaction record; express The total number of target domain interaction records in the dataset.
[0010] In one embodiment, the step of constructing corresponding text training datasets, merging the text training datasets of each source domain with the text training datasets of the target domain to form a hybrid training dataset corresponding to each source domain, specifically includes: For each source domain dataset , ,Will Items that have been interacted with by the same user are sorted in ascending order of timestamps. Each user has one sequence of interacted items, and each item is represented by its title string. The set of all user interaction sequences is named ; For the target domain dataset , to proceed and to obtain The same processing procedure yields a set of user interaction sequences for the target domain. ; for and any interaction sequence in Using the leave-one-out method, the first element in the interaction sequence is... Items Separate, used as test set labels; the remaining part For building the training set: text instructions for inputting a large model. This includes natural language descriptions of the recommendation task and user history. and candidate set, the candidate set is composed of It consists of 29 negative samples, labeled It is the correct item sorting, including real items. It is placed at the very beginning; in this way, they can be constructed separately. Corresponding source domain text training dataset and The corresponding target domain text training dataset ,in, express The Text data, express The The text data is input as text instructions to the large model. express The tag, express The total number of text data in the data; express The Text data, express The The text instructions of the bar, express The tag, express The total number of text data in the document; Training datasets of text from various source domains Training datasets with target domain text respectively The datasets are then merged to obtain a mixed training dataset corresponding to each source domain.
[0011] In one embodiment, the large model includes an embedding layer and The `transformer` submodule of the layer; definition For the first The pre-trained weight matrix of the transformer submodule of the layer. , express The length, express The width; Among them, the nth mixed training dataset corresponds to In the hybrid low-rank expert module of the layer, the first The two low-rank matrices of the layer are denoted as and , , Let be the total number of the mixed training datasets. The first layer of the target domain low-rank expert module The two low-rank matrices of the layer are denoted as and ; This represents the rank of a low-rank matrix.
[0012] In one embodiment, the step of fine-tuning the target domain low-rank expert module loaded into the large model based on the target domain text training dataset to obtain an optimized target domain low-rank expert module specifically includes: Text instructions for each text data in the target domain text training dataset The input is fed into a large model and processed by the embedding layer to obtain the initial target domain embedding representation. ; Will The inputs are respectively processed in the transformer submodule of layer L and the target domain low-rank expert module of layer L for fusion processing to obtain the first... Layer target domain embedding representation This leads to the target domain embedding representation of the Lth layer. : ; Indicates the first The target domain embedding representation of the layer; The fine-tuning optimization objective involves constructing an embedding layer, an L-layer transformer submodule, and an L-layer target domain low-rank expert module. The target domain low-rank expert module of layer L is then fine-tuned to obtain the optimal parameters of the fine-tuned target domain low-rank expert module. ,in, and express and The corresponding optimal low-rank matrix: ; Represents the parameters of a large model. = These are the parameters of the low-rank expert module of the target domain in layer L. for The tag, Indicates input Predicted as The probability of; express The total amount of text data in the data.
[0013] In one embodiment, the step of fine-tuning the hybrid low-rank expert module loaded into the large model based on each hybrid training dataset to obtain the optimized hybrid low-rank expert module corresponding to each hybrid training dataset specifically includes: For each mixed dataset Text instructions The input is fed into a large model and processed by the embedding layer to obtain the initial target domain embedding representation. ;Will The inputs are respectively processed in the transformer submodule of layer L and the target domain low-rank expert module of layer L for fusion processing to obtain the first... Layer target domain embedding representation This leads to the target domain embedding representation of the Lth layer. : ; Indicates the first The target domain embedding representation of the layer; The fine-tuning optimization objective for constructing the embedding layer, the L-layer transformer submodule, and the L-layer hybrid low-rank expert module. The hybrid low-rank expert module of layer L is then fine-tuned to obtain the optimal parameters of the fine-tuned hybrid low-rank expert module. ,in, and express and The corresponding optimal low-rank matrix: ; Represents the parameters of a large model. = It is the first The parameters of the L layer of a hybrid low-rank expert module. for The tag, Indicates input Predicted as The probability of; express The total number of text data in the document; Through the above operations, we obtain The optimal parameters of the optimized hybrid low-rank expert module corresponding to each hybrid training dataset , ,in, and Indicates the first The first hybrid low-rank expert module The two optimal low-rank matrices of the layer.
[0014] In one embodiment, the step of weightedly fusing the optimal parameters of the optimized target domain low-rank expert module with those of all optimized hybrid low-rank expert modules to obtain a fused low-rank expert module specifically includes: Based on the optimal parameters of the optimized target domain low-rank expert module and the optimal parameters of each hybrid low-rank expert module, the first... Two fusion low-rank matrices of the fusion low-rank expert module of the layer and Thus obtain Fusion parameters of low-rank expert modules in layer fusion : ; ; When i=0 For the optimized target domain low-rank expert module The two optimal low-rank matrices of the layer form the target domain low-rank expert module. The optimal parameters of the layer; when 1 hour, For the optimized hybrid low-rank expert module The two optimal low-rank matrices of the layer form the hybrid low-rank expert module. The optimal parameters of the layer; Denotes the fusion coefficient, satisfying .
[0015] In one embodiment, loading the fused low-rank expert module into the large model for inference and prediction of sequence recommendation tasks for target domain users specifically includes: Calculate the parameters of a large model that has been loaded with a fused low-rank expert module. : ; These are the original parameters of the large model. To integrate the fusion parameters of the low-rank expert module; The target domain test set is obtained through a process similar to that used to construct the target domain training dataset. ,in Indicates the first The input text instructions for the large model include natural language descriptions of the recommendation task and user history. and candidate set (composed of items) (and consist of 29 negative samples). express The corresponding correct answer, item Ranked first; Using a large model loaded with a low-rank expert module, the target domain test set was tested. Each test text instruction in Make predictions to obtain the sequence of items that each user is most likely to interact with next.
[0016] Compared with the prior art, the beneficial technical effects of the present invention are: 1. This invention proposes a novel model paradigm-based method, namely low-rank module weaving fusion, which integrates hybrid low-rank experts possessing recommendation knowledge from both the source and target domains with low-rank experts from the target domain. This allows the large model to maintain high recommendation accuracy in the target domain while also exhibiting cross-domain knowledge transfer capabilities. Useful knowledge learned from the source domain enhances recommendation accuracy in the target domain. The fusion process is based on a unified weight averaging strategy, which is simple and efficient, improving the large model's correct transfer of cross-domain recommendation knowledge and its accuracy in the target domain.
[0017] 2. This invention can generalize large-scale model recommendation to various cross-domain recommendation scenarios. In single-source and multi-source domain scenarios, it is only necessary to interweave the data from each source domain with the target domain data to construct a hybrid training dataset and train the corresponding hybrid low-rank module. This allows for effective transfer of user behavior knowledge from the source domain through weighted average fusion, alleviating the problem of data sparsity in the target domain. In cross-domain and cross-platform scenarios, this invention unifies the interaction sequence into semantic input through text commands. Combined with the lightweight adaptability of the low-rank module, it can still achieve effective knowledge transfer and improve the recommendation performance of the large model in the target domain. In addition, by leveraging the context learning capability of the large model, this invention naturally possesses task generalization ability, such as the ability to generate interpretable recommendation results.
[0018] 3. This invention is easy to use and supports plug-and-play integration. The source domain hybrid low-rank expert only needs to be trained once and can be reused repeatedly (it can still be used directly when the domain roles are interchanged), while the target domain low-rank expert can be fused after minimal fine-tuning. Moreover, the fusion method does not increase the number of additional parameters or change the model structure, enabling it to combine with a variety of existing open-source technologies. Attached Figure Description
[0019] Figure 1 This is a diagram of the overall architecture of the present invention. Detailed Implementation
[0020] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.
[0021] This embodiment presents a large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion. The aim is to enhance the cross-domain recommendation knowledge transfer capability of large models through a novel model fusion paradigm, thereby improving the recommendation accuracy of large models in the target domain. Specifically, the method proceeds as follows: Step 1: Construct the target domain text training dataset Source domain text training dataset and mixed training datasets .
[0022] Step 1 specifically includes: Step 1.1: Obtain the target domain dataset and source domain dataset , ,in, express The Source domain interaction records, Indicates the first The source domain user in the source domain interaction record, Indicates the first Source domain items in the source domain interaction log. Indicates the first The source domain timestamp in the source domain interaction record, Indicates the first The total number of source domain interaction records for each source domain dataset. Represents the first of the target domain datasets Target field interaction records, Indicates the first The target domain user in the target domain interaction record, Indicates the first Target domain items in the target domain interaction record. express The target domain timestamp in the target domain interaction record; This represents the total number of target domain interaction records in the target domain dataset.
[0023] In a preferred embodiment, this invention uses the publicly available recommendation datasets Amazon Review and MovieLens, as shown in Tables 1 and 2. Density represents the density of interactions within the dataset, calculated as the number of interactions ÷ (number of users × number of items). Both datasets contain user-item interaction records, including user ID, item ID, and interaction timestamps. The datasets also contain item information, including item ID, item title, item category, and item description. The Amazon Review dataset is cleaned using a common five-core filtering technique to remove users and items with fewer than five interaction records. For the cross-domain Amazon Review dataset, this invention uses four domains: Clothing, Footwear, and Jewelry; Beauty; Food; and Sports. For the MovieLens dataset, this invention uses one domain, containing one million interaction records, with no overlap between the different domains. For each domain in the Amazon Review dataset, this invention uses pairwise mixing of data from the four domains—Clothing, Beauty, Food, and Sports—as the source domain mixed training dataset, and uses data from the four domains—Clothing, Beauty, Food, and Sports—as the target domain datasets respectively. For the cross-platform MovieLens dataset, MovieLens-1M data is used as the source domain dataset and Toys data as the target domain dataset, and Toys data is used as the source domain dataset and MovieLens-1M data as the target domain dataset.
[0024] Table 1. Detailed data for each domain in the Amazon Review dataset:
[0025] Table 2. Detailed data for each domain of the MovieLens dataset:
[0026] Step 1.2: For each source domain , , the dataset Items from the same user's source domain are arranged sequentially by timestamp to obtain a set of item title sequences for all source domain users. .
[0027] Step 1.3: Target domain dataset Items from the same user interaction within the target domain are arranged sequentially by timestamp, resulting in a set of item title sequences for all target domain users. .
[0028] Step 1.4, according to and Construct the first Text training dataset from multiple source domains and target domain text training dataset ,in, Indicates the first The first source domain Text data, Indicates the first The first source domain A text instruction, express The tag, Indicates the first The total number of data from each source domain. Indicates the first Target domain data, Indicates the first Item field text input data, express The tag, This indicates the total number of data in the target domain.
[0029] Step 1.5: Train the text dataset based on the data obtained in the previous step. and The text training datasets from each source domain are merged with the text training dataset from the target domain to obtain the mixed training datasets corresponding to each source domain. , .
[0030] In a preferred embodiment, the input template for the text training dataset is: the user has purchased [user interaction history]. Based on the user's purchase history, please recommend 5 items from the candidate set that the user may purchase next, sorted from high to low probability. The candidate set is [candidate items]. The text output template is the sorting of the titles of the five items, with the title appearing first if the probability is higher.
[0031] Step 2: Construct large models separately and train them on the target domain text training datasets. and N mixed training datasets Fine-tuning was performed on the large model to obtain the parameters of the target domain low-rank expert modules with the same structure. And the parameters of N mixed low-rank expert modules These are all the optimal parameters during the fine-tuning process.
[0032] Step 2 specifically includes: Step 2.1: Construct the large model, which includes: embedding layer, The transformer submodule of the layer defines For the first The pre-trained weight matrix of the transformer submodule of the layer. Represents the length of the matrix. This represents the width of the matrix; in a preferred embodiment, the large model is selected from the Qwen2-7B-Instruct large model of the generalized Qwen2. , and It is determined by the specific large model and its sub-modules that are loaded.
[0033] Step 2.2, let the nth source domain be... In the low-rank expert module of the layer The two low-rank matrices of the layer are denoted as and ,Will The first layer of the target domain low-rank expert module The two low-rank matrices of the layer are denoted as and ; This represents the rank of a low-rank matrix; in a preferred embodiment, the low-rank expert is constructed using the LoRA method, and the rank... Set to 16.
[0034] Step 2.3: For the target domain text training dataset ,Will The input is fed into a large model and processed by the embedding layer to obtain the initial target domain embedding representation. .
[0035] Step 2.4 After fusion processing by inputting the transformer submodule of layer L and the target domain low-rank expert module of layer L respectively, the result is obtained using equation (1). Layer target domain embedding representation Thus, the target domain embedding representation of the Lth layer is obtained. : (1) In equation (1), Indicates the first The target domain embedding representation of the layer.
[0036] Step 2.5: Use equation (2) to construct the embedding layer, the transformer submodule of the L layer, and the target domain low-rank expert module of the L layer for fine-tuning and optimization. And fine-tune the low-rank expert module of the target domain in layer L, such as... Figure 1 As shown, during the fine-tuning process, the parameters of the embedding layer and transformer submodule of the large model are frozen and do not participate in parameter updates. Only the parameters of the source domain low-rank expert module participate in the update, thus obtaining the optimal parameters of the target domain low-rank expert module after fine-tuning. ,in, and Indicates the first The two optimal target domain low-rank matrices of the layer: (2) In equation (2), Parameters representing large language models, = These are the parameters of the low-rank expert module in the L-layer target domain. Indicates input Predicted as The probability of.
[0037] Step 2.6: Similarly, for each mixed training dataset... Following steps 2.3-2.5, the optimal parameters of the fine-tuned hybrid low-rank expert module are obtained. , ,in, and This indicates the nth hybrid low-rank expert module. The two optimal low-rank matrices of the layer.
[0038] Step 3: Construct a fusion low-rank expert module, consisting of N hybrid low-rank expert modules and a target domain low-rank expert module, and set a fusion coefficient that satisfies the condition that the sum of the weights is 1. The parameters of the fused low-rank modules are obtained by weighted summation of the low-rank matrices of the LoRA layers of each module. .
[0039] Step 3 specifically includes: Step 3.1: Optimal parameters for the aforementioned target domain low-rank expert module. Optimal parameters of several hybrid low-rank expert modules Using equations (3) and (4), we can obtain the first... Two fusion low-rank matrices of the fusion low-rank expert module of the layer and Thus obtain Fusion parameters of low-rank expert modules in layer fusion ; (3) (4) In equations (3) and (4), Denotes the fusion coefficient, satisfying The fusion coefficient can be determined through performance tuning on the target domain validation set, or set to the default value. .
[0040] Step 4: In the inference phase, utilize parameters based on fusion. Large Model Predict the items that users will interact with.
[0041] Step 4 specifically includes: Step 4.1: Construct the large model for the reasoning stage, and use equation (5) to obtain the parameters of the large model loaded with the fusion low-rank expert module. : (5) Step 4.2: Utilize the large model based on fusion parameters Predict the question text input by the user in the target domain to obtain the sequence of items the user will interact with.
[0042] In a preferred embodiment, since the merged model structure remains the same as the original large model, VLLM technology is used to accelerate inference, significantly reducing the GPU memory and time consumption of inference. During the model testing phase, the input test text training dataset is used, and the inference results of the large model are saved in output order. Each test data point is labeled, and the Normalized Decay Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR) are calculated based on the position of the labels in the output sequence to evaluate the quality of the large model's item recommendations.
[0043] The final experimental results are shown in Table 3a, Table 3b and Table 4, where the comparison methods are GRU4Rec (Session-based Recommendations with Recurrent Neural Networks), SASRec (Self-attentive sequential recommendation), FMLP-Rec (Filter-enhanced MLP is All You Need for Sequential Recommendation), MCRPL (MCRPL: A Pretrain, Prompt&Fine-tune Paradigm for Non-overlapping Many-to-one Cross-domain Recommendation), UniSRec (Towards universal sequence representation learning for recommender systems), VQ-Rec (Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders), RecFormer (Text Is All You Need: Learning Language Representations for Sequential Recommendation), Qwen2-7B (Qwen2 Technical Report), TALLRec (Tallrec: An effective and efficient tuning framework to align large language model with recommendation.The following are related technologies: LLM-Rec (Personalized Recommendation via Prompting Large Language Models), WeightAverage (Weighted average fusion of source and target domain models), AdaMerging (Adaptive Model Merging for Multi-Task Learning), LoRA-LEGO (Merging LoRAs like PlayingLEGO: Pushing the Modularity of LoRA to Extremes Through Rank-WiseClustering), Ties-Merging (TIES-Merging: Resolving Interference When Merging Models), Target-domain Only (Training the model using only target domain data), All Data Merging (Training a single model using all domain data), and WeaveRec (Using the complete steps of this invention).
[0044] Table 3a, Performance comparison in cross-domain scenarios:
[0045] Table 3b, Performance comparison in cross-domain scenarios:
[0046] Table 4. Performance comparison in cross-platform scenarios:
[0047] In Tables 3a, 3b and 4, the underlined data represents the suboptimal results, and the bolded data represents the optimal results. It can be seen that the present invention has achieved good results in multiple fields in cross-domain and cross-platform scenarios. These experimental results fully demonstrate the effectiveness of the method of the present invention, indicating that the present invention can effectively alleviate cross-domain knowledge conflicts and improve the performance of large models in the target domain.
[0048] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.
[0049] In one embodiment, the present invention provides a computer-readable storage medium including instructions, such as a memory including instructions that can be executed by a processor to perform the above-described method. The storage medium may be a computer-readable storage medium, for example, a ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0050] In one embodiment, the present invention provides a computer device, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.
[0051] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0052] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.
[0053] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A cross-domain sequence recommendation method for large models based on low-rank module weaving fusion, characterized in that, include: Obtain the target domain dataset and multiple source domain datasets, and construct corresponding text training datasets for each. Merge the text training datasets of each source domain with the text training datasets of the target domain to form a mixed training dataset for each source domain. Each interaction record in the target domain dataset and the source domain dataset includes the user, the item that the user interacted with, and the interaction timestamp; Based on the target domain text training dataset, the target domain low-rank expert module loaded into the large model is fine-tuned to obtain the optimized target domain low-rank expert module. Based on each hybrid training dataset, the hybrid low-rank expert module loaded into the large model is fine-tuned to obtain the optimized hybrid low-rank expert module corresponding to each hybrid training dataset. The optimal parameters of the optimized target domain low-rank expert module and all optimized hybrid low-rank expert modules are weighted and fused to obtain the fused low-rank expert module. The low-rank expert module is loaded into the large model for inference and prediction of sequence recommendation tasks for target domain users.
2. The method for cross-domain sequence recommendation in a large model based on low-rank module weaving fusion according to claim 1, characterized in that, The acquisition of the target domain dataset and multiple source domain datasets includes: Obtain the target domain dataset and the nth source domain dataset , , The total number of data sets in the source domain; where, express The Source domain interaction records, Indicates the first The source domain user in the source domain interaction record, Indicates the first Source domain items in the source domain interaction log. Indicates the first The source domain timestamp in the source domain interaction record; Indicates the first The total number of source domain interaction records in each source domain dataset; Represents the first of the target domain datasets Target field interaction records, Indicates the first The target domain user in the target domain interaction record, Indicates the first Target domain items in the target domain interaction record. express The target domain timestamp in the target domain interaction record; express The total number of target domain interaction records in the dataset.
3. The method for cross-domain sequence recommendation of a large model based on low-rank module weaving fusion according to claim 2, characterized in that, The step of constructing corresponding text training datasets and merging the text training datasets of each source domain with the text training datasets of the target domain to form a mixed training dataset for each source domain specifically includes: For each source domain dataset , ,Will Items that have been interacted with by the same user are sorted in ascending order of timestamps. Each user has one sequence of interacted items, and each item is represented by its title string. The set of all user interaction sequences is named ; For the target domain dataset , to proceed and to obtain The same processing procedure yields a set of user interaction sequences for the target domain. ; for and any interaction sequence in Using the leave-one-out method, the first element in the interaction sequence is... Items Separate, used as test set labels; the remaining part For building the training set: text instructions for inputting a large model. This includes natural language descriptions of the recommendation task and user history. and candidate set, the candidate set is composed of It consists of 29 negative samples, labeled It is the correct item sorting, including real items. It is placed at the very beginning; in this way, they can be constructed separately. Corresponding source domain text training dataset and The corresponding target domain text training dataset ,in, express The Text data, express The The text data is input as text instructions to the large model. express The tag, express The total number of text data in the data; express The Text data, express The The text instructions of the bar, express The tag, express The total number of text data in the document; Training datasets of text from various source domains Training datasets with target domain text respectively The datasets are then merged to obtain a mixed training dataset corresponding to each source domain.
4. The large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion according to claim 1, characterized in that, The large model includes an embedding layer and The `transformer` submodule of the layer; definition For the first The pre-trained weight matrix of the transformer submodule of the layer. , express The length, express The width; Among them, the nth mixed training dataset corresponds to In the hybrid low-rank expert module of the layer, the first The two low-rank matrices of the layer are denoted as and , , Let be the total number of the mixed training datasets. The first layer of the target domain low-rank expert module The two low-rank matrices of the layer are denoted as and ; This represents the rank of a low-rank matrix.
5. The large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion according to claim 4, characterized in that, The target domain low-rank expert module, loaded into the large model based on the target domain text training dataset, is fine-tuned to obtain an optimized target domain low-rank expert module, specifically including: Text instructions for each text data in the target domain text training dataset The input is fed into a large model and processed by the embedding layer to obtain the initial target domain embedding representation. ; Will The inputs are respectively processed in the transformer submodule of layer L and the target domain low-rank expert module of layer L for fusion processing to obtain the first... Layer target domain embedding representation This leads to the target domain embedding representation of the Lth layer. : ； Indicates the first The target domain embedding representation of the layer; The fine-tuning optimization objective involves constructing an embedding layer, an L-layer transformer submodule, and an L-layer target domain low-rank expert module. The target domain low-rank expert module of layer L is then fine-tuned to obtain the optimal parameters of the fine-tuned target domain low-rank expert module. ,in, and express and The corresponding optimal low-rank matrix: ； Represents the parameters of a large model. = These are the parameters of the low-rank expert module of the target domain in layer L. for The tag, Indicates input Predicted as The probability of; express The total amount of text data in the data.
6. The large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion according to claim 4, characterized in that, The process involves fine-tuning the hybrid low-rank expert module loaded into the large model based on each hybrid training dataset, resulting in an optimized hybrid low-rank expert module for each hybrid training dataset. Specifically, this includes: For each mixed dataset Text instructions The input is fed into a large model and processed by the embedding layer to obtain the initial target domain embedding representation. ;Will The inputs are respectively processed in the transformer submodule of layer L and the target domain low-rank expert module of layer L for fusion processing to obtain the first... Layer target domain embedding representation This leads to the target domain embedding representation of the Lth layer. : ； Indicates the first The target domain embedding representation of the layer; The fine-tuning optimization objective for constructing the embedding layer, the L-layer transformer submodule, and the L-layer hybrid low-rank expert module. The hybrid low-rank expert module of layer L is then fine-tuned to obtain the optimal parameters of the fine-tuned hybrid low-rank expert module. ,in, and express and The corresponding optimal low-rank matrix: ； Represents the parameters of a large model. = It is the first The parameters of the L layer of a hybrid low-rank expert module. for The tag, Indicates input Predicted as The probability of; express The total number of text data in the document; Through the above operations, we obtain The optimal parameters of the optimized hybrid low-rank expert module corresponding to each hybrid training dataset , ,in, and Indicates the first The first hybrid low-rank expert module The two optimal low-rank matrices of the layer.
7. The large-scale cross-domain sequence recommendation method based on low-rank module weaving fusion according to claim 1, characterized in that, The step of weightedly fusing the optimal parameters of the optimized target domain low-rank expert module with those of all optimized hybrid low-rank expert modules to obtain a fused low-rank expert module specifically includes: Based on the optimal parameters of the optimized target domain low-rank expert module and the optimal parameters of each hybrid low-rank expert module, the first... Two fusion low-rank matrices of the fusion low-rank expert module of the layer and Thus obtain Fusion parameters of low-rank expert modules in layer fusion : ； ； When i=0 For the optimized target domain low-rank expert module The two optimal low-rank matrices of the layer form the target domain low-rank expert module. The optimal parameters of the layer; when 1 hour, For the optimized hybrid low-rank expert module The two optimal low-rank matrices of the layer form the hybrid low-rank expert module. The optimal parameters of the layer; Denotes the fusion coefficient, satisfying .
8. The method for cross-domain sequence recommendation in a large model based on low-rank module weaving fusion according to claim 1, characterized in that, The step of loading the low-rank expert module into the large model for inference and prediction of sequence recommendation tasks for target domain users specifically includes: Calculate the parameters of a large model that has been loaded with a fused low-rank expert module. : ； These are the original parameters of the large model. To integrate the fusion parameters of the low-rank expert module; By using a large model loaded with a low-rank expert module, the test text instructions in the test set are predicted to obtain the sequence of items that the user is most likely to interact with next.