A method and system for optimizing and adjusting large model tasks based on LoRA

Abstract

The present invention is applicable to the technical field of large-model task optimization and adjustment, and provides a large-model task optimization and adjustment method and system based on LoRA. The present invention determines multiple downstream tasks, matches multiple corresponding LoRA modules, and initializes the matrix parameters of multiple LoRA modules by performing application task monitoring; performs parameter training on multiple LoRA modules and records parameter data; uses a sentence embedding model to determine the embedding representation of multiple LoRA modules; calculates the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculates the optimized adjustment output based on the multiple contribution weights. It can achieve more flexible and adaptable fine-tuning for multiple tasks, dynamically balance resource allocation and performance requirements between tasks, effectively cope with the complexity and heterogeneity in the multi-task environment, and meet the personalized requirements of the multi-task system, improving the model reasoning efficiency and overall performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of large model task optimization and adjustment, and in particular relates to a large model task optimization and adjustment method and system based on LoRA. Background Art

[0002] With the widespread deployment of large language models in various real-world applications, user needs are no longer limited to single tasks. More and more scenarios involve cross-domain, multi-task requests. For example, in applications such as smart homes, autonomous driving, and medical diagnosis, users may request complex and random mixed tasks, which places higher demands on the model's generalization capabilities, responsiveness, and resource scheduling.

[0003] LoRA, as a method for efficient parameter fine-tuning, offers a potential solution. By freezing most model parameters and adjusting only specific low-rank matrices, LoRA allows for flexible adaptation to specific task scenarios at a low computational cost. However, existing LoRA applications mostly employ a single fine-tuning strategy, which struggles to adequately address the complexity and heterogeneity of multi-task environments. Furthermore, the performance requirements of different tasks vary significantly, making a single global fine-tuning approach difficult to meet the personalized requirements of multi-task systems. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a large model task optimization and adjustment method and system based on LoRA, aiming to solve the technical problems existing in the existing technology mentioned in the background technology.

[0005] The embodiment of the present invention is implemented as follows:

[0006] A large model task optimization and adjustment method based on LoRA, the method specifically comprising the following steps:

[0007] Perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize the matrix parameters of multiple LoRA modules;

[0008] Obtain data sets for multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data;

[0009] Obtaining multiple prompts for the downstream tasks, using a sentence embedding model to map the multiple prompts and corresponding LoRA modules to the same embedding space, and determining the embedding representations of the multiple LoRA modules;

[0010] Calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights.

[0011] As a further limitation of the technical solution of the embodiment of the present invention, the application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of the multiple LoRA modules specifically include the following steps:

[0012] Conduct application task monitoring and identify multiple downstream tasks;

[0013] According to the plurality of downstream tasks, matching a plurality of corresponding LoRA modules;

[0014] Creating a LoRA structure of multiple LoRA modules;

[0015] Initialize the matrix parameters of the LoRA structure of multiple LoRA modules.

[0016] As a further limitation of the technical solution of the embodiment of the present invention, the LoRA structure is composed of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, is the output dimension.

[0017] As a further limitation of the technical solution of the embodiment of the present invention, in the matrix parameters of the LoRA structure of the initialization multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to Gaussian distribution.

[0018] As a further limitation of the technical solution of the embodiment of the present invention, the obtaining of the data sets of the plurality of downstream tasks, performing parameter training on the plurality of LoRA modules, and recording the parameter data specifically include the following steps:

[0019] Obtaining a plurality of datasets for the downstream tasks;

[0020] Based on the plurality of data sets, constructing loss functions corresponding to the plurality of LoRA modules and the plurality of downstream tasks;

[0021] According to the multiple loss functions, parameter training is performed on the multiple LoRA modules, and parameter data is recorded.

[0022] As a further limitation of the technical solution of the embodiment of the present invention, the loss function is:

[0023] ;

[0024] ;

[0025] in, Indicates the Downstream tasks, Indicates the LoRA modules, Indicates the datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0026] As a further limitation of the technical solution of the embodiment of the present invention, the embedding of multiple LoRA modules is expressed as:

[0027] ;

[0028] in, Indicates the The embedded representation of a LoRA module, Indicates a task No. The embedding representation of domain-specific samples, A sample of a specific area.

[0029] As a further limitation of the technical solution of the embodiment of the present invention, the calculating of contribution weights between multiple LoRA modules and multiple downstream tasks, and calculating the optimized adjustment output according to the multiple contribution weights specifically include the following steps:

[0030] Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks;

[0031] Based on the multiple cosine similarities, calculating the contribution weights of the multiple LoRA modules;

[0032] An optimized adjustment output is calculated based on the plurality of contribution weights.

[0033] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula of the multiple cosine similarities is:

[0034] ;

[0035] in, Indicates the The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm;

[0036] The calculation formula for the multiple contribution weights is:

[0037] ;

[0038] in, Indicates the The contribution weights corresponding to the LoRA modules are LoRA modules;

[0039] The calculation formula of the optimization adjustment output is:

[0040] .

[0041] A large model task optimization and adjustment system based on LoRA, the system includes a LoRA module creation unit, a LoRA parameter training unit, a LoRA embedding analysis unit and an optimization output processing unit, wherein:

[0042] A LoRA module creation unit is used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize the matrix parameters of multiple LoRA modules;

[0043] A LoRA parameter training unit is used to obtain a plurality of data sets of downstream tasks, perform parameter training on a plurality of LoRA modules, and record parameter data;

[0044] A LoRA embedding analysis unit is used to obtain prompts of multiple downstream tasks, map multiple prompts and corresponding LoRA modules to the same embedding space using a sentence embedding model, and determine the embedding representations of multiple LoRA modules;

[0045] The optimization output processing unit is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The embodiment of the present invention monitors application tasks, determines multiple downstream tasks, matches multiple corresponding LoRA modules, and initializes the matrix parameters of multiple LoRA modules; performs parameter training on multiple LoRA modules and records parameter data; uses a sentence embedding model to determine the embedding representation of multiple LoRA modules; calculates the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculates and optimizes the adjustment output based on the multiple contribution weights. It can achieve more flexible and adaptable fine-tuning for multiple tasks, dynamically balance resource allocation and performance requirements between tasks, effectively cope with the complexity and heterogeneity in multi-task environments, and meet the personalized requirements of multi-task systems, improving model reasoning efficiency and overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a large model task optimization and adjustment method based on LoRA provided by an embodiment of the present invention is shown;

[0049] Figure 2 A flowchart of matching multiple LoRA modules in a method provided in an embodiment of the present invention is shown;

[0050] Figure 3 A flowchart of parameter training for multiple LoRA modules in the method provided in an embodiment of the present invention is shown;

[0051] Figure 4 A flowchart of calculating optimization and adjustment output in the method provided by an embodiment of the present invention is shown;

[0052] Figure 5 The application architecture diagram of the LoRA-based large model task optimization and adjustment system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] It is understandable that LoRA, as a parameter-efficient fine-tuning method, provides a potential solution. By freezing most model parameters and adjusting only specific low-rank matrices, LoRA can flexibly adapt to specific task scenarios at a low computational cost. However, most LoRA applications in existing technologies use a single fine-tuning strategy, which is difficult to fully cope with the complexity and heterogeneity of multi-task environments. In addition, different tasks have significant differences in their requirements for model performance, and a single global fine-tuning strategy cannot meet the personalized requirements of multi-task systems.

[0055] To solve the above problems, the embodiment of the present invention discloses a large-model task optimization and adjustment method and system based on LoRA. By monitoring application tasks, multiple downstream tasks are determined, multiple corresponding LoRA modules are matched, and the matrix parameters of multiple LoRA modules are initialized; data sets of multiple downstream tasks are obtained, parameter training of multiple LoRA modules is performed, and parameter data is recorded; prompts of multiple downstream tasks are obtained, and sentence embedding models are used to map multiple prompts and corresponding LoRA modules to the same embedding space, and embedding representations of multiple LoRA modules are determined; contribution weights between multiple LoRA modules and multiple downstream tasks are calculated, and optimization and adjustment outputs are calculated based on the multiple contribution weights. It can achieve more flexible and adaptable fine-tuning for multiple tasks, dynamically balance resource allocation and performance requirements between tasks, effectively cope with the complexity and heterogeneity in multi-task environments, meet the personalized requirements of multi-task systems, and improve model reasoning efficiency and overall performance.

[0056] Specifically, Figure 1 A flowchart of a LoRA-based large model task optimization and adjustment method provided in an embodiment of the present invention is shown.

[0057] In a preferred embodiment provided by the present invention, a large model task optimization and adjustment method based on LoRA, the method specifically comprises the following steps:

[0058] Step S101: perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules.

[0059] In an embodiment of the present invention, the large language model needs to adapt to a variety of different downstream tasks. Different downstream tasks have significant heterogeneity in many aspects. By performing application task monitoring, multiple downstream tasks are determined, and then multiple corresponding LoRA modules are matched according to the multiple downstream tasks. By creating a LoRA structure of multiple LoRA modules, the matrix parameters of the multiple LoRA modules are initialized based on the LoRA structure. Specifically, the LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, For the output dimension, initialize the matrix parameters of the LoRA structure of multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to Gaussian distribution.

[0060] It is understandable that multiple downstream tasks have different task types. In the embodiment of the present invention, the different task types include: text classification, machine translation, question answering system, sentiment analysis, text summarization and named entity recognition, etc.

[0061] Specifically, Figure 2 A flowchart of matching multiple LoRA modules in the method provided by an embodiment of the present invention is shown.

[0062] Among them, in the preferred embodiment provided by the present invention, the application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules specifically include the following steps:

[0063] Step S1011: perform application task monitoring and determine multiple downstream tasks.

[0064] Step S1012: Match multiple corresponding LoRA modules according to the multiple downstream tasks.

[0065] Step S1013: Create a LoRA structure of multiple LoRA modules.

[0066] Step S1014: Initialize the matrix parameters of the LoRA structure of the multiple LoRA modules.

[0067] Furthermore, the LoRA-based large model task optimization and adjustment method further includes the following steps:

[0068] Step S102: Acquire data sets for multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data.

[0069] In an embodiment of the present invention, by obtaining data sets of multiple downstream tasks, based on multiple data sets, keeping the main model parameters unchanged, training the corresponding LoRA modules, constructing loss functions corresponding to multiple LoRA modules and multiple downstream tasks, and then performing parameter training on multiple LoRA modules according to multiple loss functions, and recording parameter data. Specifically, the loss function is:

[0070] ;

[0071] ;

[0072] in, Indicates the Downstream tasks, Indicates the LoRA modules, Indicates the datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0073] Specifically, Figure 3 A flowchart of parameter training for multiple LoRA modules in the method provided by an embodiment of the present invention is shown.

[0074] Among them, in the preferred embodiment provided by the present invention, the obtaining of data sets of multiple downstream tasks, performing parameter training on multiple LoRA modules, and recording parameter data specifically include the following steps:

[0075] Step S1021: Acquire a plurality of data sets of the downstream tasks.

[0076] Step S1022: Based on the multiple data sets, construct loss functions corresponding to the multiple LoRA modules and the multiple downstream tasks.

[0077] Step S1023: Perform parameter training on the multiple LoRA modules according to the multiple loss functions, and record parameter data.

[0078] Furthermore, the LoRA-based large model task optimization and adjustment method further includes the following steps:

[0079] Step S103: Obtain multiple Prompts of the downstream tasks, use the sentence embedding model to map the multiple Prompts and corresponding LoRA modules to the same embedding space, and determine the embedding representations of the multiple LoRA modules.

[0080] In an embodiment of the present invention, by obtaining the prompts of multiple downstream tasks, and then using the sentence embedding model, the prompts of each downstream task are converted into an embedded representation, and the embedded representations of the samples of different specific fields corresponding to the multiple downstream tasks are determined. Then, the multiple prompts and the corresponding LoRA modules are mapped to the same embedding space, and the embedded representations of the multiple LoRA modules are determined. Specifically, the embedded representations of the multiple LoRA modules are:

[0081] ;

[0082] in, Indicates the The embedded representation of a LoRA module, Indicates a task No. The embedding representation of domain-specific samples, A sample of a specific area.

[0083] Step S104: Calculate the contribution weights between the multiple LoRA modules and the multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights.

[0084] In an embodiment of the present invention, by calculating the cosine similarity between multiple LoRA modules and multiple downstream tasks, and then calculating the contribution weights of multiple LoRA modules based on the multiple cosine similarities, and then calculating the optimization adjustment output according to the multiple contribution weights, the most suitable current task and LoRA module are dynamically selected to achieve the optimization adjustment of the model. Specifically, the calculation formula of multiple cosine similarities is:

[0085] ;

[0086] in, Indicates the The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm;

[0087] The calculation formula for multiple contribution weights is:

[0088] ;

[0089] in, Indicates the The contribution weights corresponding to the LoRA modules are LoRA modules;

[0090] The calculation formula for the optimized adjustment output is:

[0091] .

[0092] Specifically, Figure 4 A flow chart of calculating the optimization adjustment output in the method provided by an embodiment of the present invention is shown.

[0093] Among them, in the preferred embodiment provided by the present invention, the calculation of the contribution weights between multiple LoRA modules and multiple downstream tasks, and the calculation of the optimization adjustment output according to the multiple contribution weights specifically include the following steps:

[0094] Step S1041: Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks.

[0095] Step S1042: Calculate contribution weights of the multiple LoRA modules based on the multiple cosine similarities.

[0096] Step S1043: Calculate the optimized adjustment output according to the multiple contribution weights.

[0097] Further, Figure 5 The application architecture diagram of the LoRA-based large model task optimization and adjustment system provided by an embodiment of the present invention is shown.

[0098] Among them, in another preferred embodiment provided by the present invention, a large model task optimization and adjustment system based on LoRA includes:

[0099] The LoRA module creation unit 101 is used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules.

[0100] In an embodiment of the present invention, the large language model needs to adapt to a variety of different downstream tasks. Different downstream tasks have significant heterogeneity in many aspects. The LoRA module creation unit 101 determines multiple downstream tasks by performing application task monitoring, and then matches multiple corresponding LoRA modules according to the multiple downstream tasks. By creating a LoRA structure of multiple LoRA modules, the matrix parameters of the multiple LoRA modules are initialized based on the LoRA structure. Specifically, the LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, For the output dimension, initialize the matrix parameters of the LoRA structure of multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to Gaussian distribution.

[0101] The LoRA parameter training unit 102 is used to obtain data sets of multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data.

[0102] In an embodiment of the present invention, the LoRA parameter training unit 102 obtains data sets of multiple downstream tasks, keeps the main model parameters unchanged based on multiple data sets, trains the corresponding LoRA modules, constructs loss functions corresponding to multiple LoRA modules and multiple downstream tasks, and then performs parameter training on multiple LoRA modules according to multiple loss functions, and records parameter data. Specifically, the loss function is:

[0103] ;

[0104] ;

[0105] in, Indicates the Downstream tasks, Indicates the LoRA modules, Indicates the datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0106] The LoRA embedding analysis unit 103 is used to obtain the prompts of multiple downstream tasks, use the sentence embedding model to map the multiple prompts and corresponding LoRA modules to the same embedding space, and determine the embedding representation of the multiple LoRA modules.

[0107] In an embodiment of the present invention, the LoRA embedding analysis unit 103 obtains the prompts of multiple downstream tasks, and then uses the sentence embedding model to convert the prompts of each downstream task into an embedded representation, determines the embedded representations of the multiple downstream tasks corresponding to different specific domain samples, and then maps the multiple prompts and corresponding LoRA modules to the same embedding space, and determines the embedded representations of the multiple LoRA modules. Specifically, the embedded representations of the multiple LoRA modules are:

[0108] ;

[0109] in, Indicates the The embedded representation of a LoRA module, Indicates a task No. The embedding representation of domain-specific samples, A sample of a specific area.

[0110] The optimization output processing unit 104 is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimization adjustment output based on the multiple contribution weights.

[0111] In an embodiment of the present invention, the optimization output processing unit 104 calculates the cosine similarities between multiple LoRA modules and multiple downstream tasks, and then calculates the contribution weights of multiple LoRA modules based on the multiple cosine similarities. Then, according to the multiple contribution weights, the optimization adjustment output is calculated, and the most appropriate current task and LoRA module are dynamically selected to achieve optimization adjustment of the model. Specifically, the calculation formula of multiple cosine similarities is:

[0112] ;

[0113] in, Indicates the The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm;

[0114] The calculation formula for multiple contribution weights is:

[0115] ;

[0116] in, Indicates the The contribution weights corresponding to the LoRA modules are LoRA modules;

[0117] The calculation formula for the optimized adjustment output is:

[0118] .

[0119] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0120] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0121] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A large model task optimization and adjustment method based on LoRA, characterized in that: The method specifically comprises the following steps: Perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize the matrix parameters of multiple LoRA modules; Obtain data sets for multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data; Obtaining multiple prompts for the downstream tasks, using a sentence embedding model to map the multiple prompts and corresponding LoRA modules to the same embedding space, and determining the embedding representations of the multiple LoRA modules; Calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights; The application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules specifically include the following steps: Conduct application task monitoring and identify multiple downstream tasks of different types, including text classification, machine translation, question answering, sentiment analysis, text summarization, and named entity recognition; According to the plurality of downstream tasks, matching a plurality of corresponding LoRA modules; Creating a LoRA structure of multiple LoRA modules; Initialize the matrix parameters of the LoRA structure of multiple LoRA modules; The LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, is the output dimension; Calculating the contribution weights between the multiple LoRA modules and the multiple downstream tasks, and calculating the optimization adjustment output according to the multiple contribution weights specifically include the following steps: Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks; Based on the multiple cosine similarities, calculating the contribution weights of the multiple LoRA modules; Calculating an optimized adjustment output according to the plurality of contribution weights; The calculation formula of the multiple cosine similarities is: ； in, Indicates the The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; Indicates the Embedded representation of a LoRA module; The calculation formula for the multiple contribution weights is: ； in, Indicates the The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula of the optimization adjustment output is: ； in, are the main model parameters and remain unchanged.

2. The large model task optimization and adjustment method based on LoRA according to claim 1 is characterized in that: In the matrix parameters of the LoRA structure of the initialization multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to Gaussian distribution.

3. The large model task optimization and adjustment method based on LoRA according to claim 1 is characterized in that: The step of obtaining a plurality of data sets of the downstream tasks, performing parameter training on the plurality of LoRA modules, and recording parameter data specifically includes the following steps: Obtaining a plurality of datasets for the downstream tasks; Based on the plurality of data sets, constructing loss functions corresponding to the plurality of LoRA modules and the plurality of downstream tasks; According to the multiple loss functions, parameter training is performed on the multiple LoRA modules, and parameter data is recorded.

4. The LoRA-based large model task optimization and adjustment method according to claim 3 is characterized in that: The loss function is: ； ； in, Indicates the Downstream tasks, Indicates the LoRA modules, Indicates the datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label.

5. The LoRA-based large model task optimization and adjustment method according to claim 4 is characterized in that: The embedding of multiple LoRA modules is represented as follows: ； in, Indicates a task No. The embedding representation of domain-specific samples, A sample of a specific area.

6. A large model task optimization and adjustment system based on LoRA, characterized in that: The system includes a LoRA module creation unit, a LoRA parameter training unit, a LoRA embedding analysis unit and an optimization output processing unit, wherein: A LoRA module creation unit is used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize the matrix parameters of multiple LoRA modules; A LoRA parameter training unit is used to obtain a plurality of data sets of downstream tasks, perform parameter training on a plurality of LoRA modules, and record parameter data; A LoRA embedding analysis unit is used to obtain prompts of multiple downstream tasks, map multiple prompts and corresponding LoRA modules to the same embedding space using a sentence embedding model, and determine the embedding representations of multiple LoRA modules; An optimization output processing unit is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights; The application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules are specifically as follows: Conduct application task monitoring and identify multiple downstream tasks of different types, including text classification, machine translation, question answering, sentiment analysis, text summarization, and named entity recognition; According to the plurality of downstream tasks, matching a plurality of corresponding LoRA modules; Creating a LoRA structure of multiple LoRA modules; Initialize the matrix parameters of the LoRA structure of multiple LoRA modules; The LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, is the output dimension; The contribution weights between the multiple LoRA modules and the multiple downstream tasks are calculated, and the optimization adjustment output is calculated based on the multiple contribution weights: Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks; Based on the multiple cosine similarities, calculating the contribution weights of the multiple LoRA modules; Calculating an optimized adjustment output according to the plurality of contribution weights; The calculation formula of the multiple cosine similarities is: ； in, Indicates the The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; Indicates the Embedded representation of a LoRA module; The calculation formula for the multiple contribution weights is: ； in, Indicates the The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula of the optimization adjustment output is: ； in, are the main model parameters and remain unchanged.

Images