LoRA-based large model task optimization adjustment method and system

Abstract

The invention is suitable for the technical field of large model task optimization and adjustment, and provides a large model task optimization and adjustment method and system based on LoRA. According to the method, multiple downstream tasks are determined through application task monitoring, multiple corresponding LoRA modules are matched, and matrix parameters of the multiple LoRA modules are initialized; performing parameter training on the plurality of LoRA modules, and recording parameter data; using a sentence embedding model to determine embedding representations of a plurality of LoRA modules; and calculating contribution weights between the plurality of LoRA modules and the plurality of downstream tasks, and calculating optimization adjustment output according to the plurality of contribution weights. According to the method, resource allocation and performance requirements between fine adjustment and dynamic balancing tasks which are more flexible and high in adaptability can be achieved for multiple tasks, complexity and heterogeneity in a multi-task environment can be effectively dealt with, personalized requirements of a multi-task system can be met, and model reasoning efficiency and overall performance are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of large model task optimization and adjustment, and in particular, relates to a large model task optimization and adjustment method and system based on LoRA. Background Art

[0002] With the widespread deployment of large language models in various practical applications, user needs are no longer limited to a single task, and more and more scenarios involve cross-domain and multi-task requests. For example, in different applications such as smart homes, autonomous driving, and medical diagnosis, users may put forward complex and random mixed task requirements, which puts higher requirements on the model's generalization ability, response speed, and resource scheduling.

[0003] As a parameter-efficient fine-tuning method, LoRA provides a potential solution. LoRA freezes most model parameters and only adjusts specific low-rank matrices, thereby flexibly adapting to specific task scenarios at a low computational cost. However, most LoRA applications in existing technologies use a single fine-tuning strategy, which is difficult to fully cope with the complexity and heterogeneity in multi-task environments. In addition, different tasks have significant differences in their requirements for model performance, and a single global fine-tuning is difficult to meet the personalized requirements of multi-task systems. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a large model task optimization and adjustment method and system based on LoRA, aiming to solve the technical problems existing in the prior art mentioned in the background technology.

[0005] The embodiment of the present invention is implemented as follows: A large model task optimization and adjustment method based on LoRA, the method specifically comprises the following steps: Perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules; Obtain data sets of multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data; Obtaining multiple prompts of the downstream tasks, using a sentence embedding model, mapping the multiple prompts and corresponding LoRA modules to the same embedding space, and determining the embedding representations of the multiple LoRA modules; Calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights.

[0006] As a further limitation of the technical solution of the embodiment of the present invention, the application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules specifically include the following steps: Conduct application task monitoring and identify multiple downstream tasks; According to the plurality of downstream tasks, matching a plurality of corresponding LoRA modules; Creating a LoRA structure of multiple LoRA modules; Initialize the matrix parameters of the LoRA structure of the plurality of LoRA modules.

[0007] As a further limitation of the technical solution of the embodiment of the present invention, the LoRA structure is composed of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, is the output dimension.

[0008] As a further limitation of the technical solution of the embodiment of the present invention, in the matrix parameters of the LoRA structure of the initialization multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to a Gaussian distribution.

[0009] As a further limitation of the technical solution of the embodiment of the present invention, the obtaining of data sets of multiple downstream tasks, performing parameter training on multiple LoRA modules, and recording parameter data specifically include the following steps: Acquire data sets of multiple downstream tasks; Based on the multiple data sets, construct loss functions corresponding to the multiple LoRA modules and the multiple downstream tasks; According to the multiple loss functions, parameter training is performed on the multiple LoRA modules, and parameter data is recorded.

[0010] As a further limitation of the technical solution of the embodiment of the present invention, the loss function is: ; ; in, Indicates Downstream tasks, Indicates LoRA modules, Indicates Datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0011] As a further limitation of the technical solution of the embodiment of the present invention, the embedding of multiple LoRA modules is expressed as: ; in, Indicates The embedded representation of a LoRA module, Indicates the task No. The embedding representation of domain-specific samples is A sample of a specific area.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the calculation of contribution weights between multiple LoRA modules and multiple downstream tasks, and the calculation of the optimization adjustment output according to the multiple contribution weights specifically include the following steps: Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks; Based on the multiple cosine similarities, calculating the contribution weights of the multiple LoRA modules; An optimized adjustment output is calculated based on the plurality of contribution weights.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formulas of the multiple cosine similarities are: ; in, Indicates The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; The calculation formula of the multiple contribution weights is: ; in, Indicates The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula of the optimization adjustment output is: .

[0014] A large model task optimization and adjustment system based on LoRA, the system includes a LoRA module creation unit, a LoRA parameter training unit, a LoRA embedding analysis unit and an optimization output processing unit, wherein: A LoRA module creation unit, used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules; A LoRA parameter training unit, used to obtain a plurality of data sets of the downstream tasks, perform parameter training on the plurality of LoRA modules, and record parameter data; A LoRA embedding analysis unit, used to obtain prompts of multiple downstream tasks, map multiple prompts and corresponding LoRA modules to the same embedding space using a sentence embedding model, and determine the embedding representations of multiple LoRA modules; The optimized output processing unit is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output according to the multiple contribution weights.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention determines multiple downstream tasks, matches multiple corresponding LoRA modules, and initializes the matrix parameters of multiple LoRA modules by performing application task monitoring; performs parameter training on multiple LoRA modules and records parameter data; uses a sentence embedding model to determine the embedding representation of multiple LoRA modules; calculates the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculates the optimized adjustment output according to the multiple contribution weights. It can achieve more flexible and adaptable fine-tuning for multiple tasks, dynamically balance resource allocation and performance requirements between tasks, effectively cope with the complexity and heterogeneity in a multi-task environment, and meet the personalized requirements of a multi-task system, improving the model reasoning efficiency and overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a large model task optimization and adjustment method based on LoRA provided in an embodiment of the present invention is shown; Figure 2 A flowchart of matching multiple LoRA modules in a method provided in an embodiment of the present invention is shown; Figure 3 A flow chart showing parameter training of multiple LoRA modules in the method provided in an embodiment of the present invention is shown; Figure 4 A flow chart showing the calculation optimization adjustment output in the method provided by an embodiment of the present invention is shown; Figure 5 The application architecture diagram of the LoRA-based large model task optimization and adjustment system provided in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] It is understandable that LoRA, as a parameter-efficient fine-tuning method, provides a potential solution. LoRA freezes most model parameters and only adjusts specific low-rank matrices, thereby flexibly adapting to specific task scenarios at a low computational cost. However, most LoRA applications in the existing technology use a single fine-tuning strategy, which is difficult to fully cope with the complexity and heterogeneity in multi-task environments. In addition, different tasks have significant differences in their requirements for model performance, and a single global fine-tuning is difficult to meet the personalized requirements of multi-task systems.

[0019] To solve the above problems, the embodiment of the present invention discloses a large model task optimization and adjustment method and system based on LoRA, which determines multiple downstream tasks by performing application task monitoring, matches multiple corresponding LoRA modules, and initializes the matrix parameters of multiple LoRA modules; obtains data sets of multiple downstream tasks, performs parameter training on multiple LoRA modules, and records parameter data; obtains prompts of multiple downstream tasks, uses sentence embedding models, maps multiple prompts and corresponding LoRA modules to the same embedding space, and determines the embedding representation of multiple LoRA modules; calculates the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculates the optimization and adjustment output according to multiple contribution weights. It can achieve more flexible and adaptable fine-tuning for multiple tasks, dynamically balance resource allocation and performance requirements between tasks, effectively cope with the complexity and heterogeneity in multi-task environments, and meet the personalized requirements of multi-task systems, improving model reasoning efficiency and overall performance.

[0020] Specifically, Figure 1 A flowchart of a LoRA-based large model task optimization and adjustment method provided in an embodiment of the present invention is shown.

[0021] In a preferred embodiment of the present invention, a large model task optimization and adjustment method based on LoRA, the method specifically comprises the following steps: Step S101: perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules.

[0022] In an embodiment of the present invention, the large language model needs to adapt to a variety of different downstream tasks. Different downstream tasks have significant heterogeneity in many aspects. By performing application task monitoring, multiple downstream tasks are determined, and then multiple corresponding LoRA modules are matched according to the multiple downstream tasks. By creating a LoRA structure of multiple LoRA modules, the matrix parameters of the multiple LoRA modules are initialized based on the LoRA structure. Specifically, the LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, For the output dimension, initialize the matrix parameters of the LoRA structure of multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to a Gaussian distribution.

[0023] It is understandable that multiple downstream tasks have different task types. In the embodiment of the present invention, the different task types include: text classification, machine translation, question answering system, sentiment analysis, text summarization and named entity recognition, etc.

[0024] Specifically, Figure 2 A flow chart of matching multiple LoRA modules in the method provided in an embodiment of the present invention is shown.

[0025] Among them, in the preferred embodiment provided by the present invention, the application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules specifically include the following steps: Step S1011: perform application task monitoring and determine multiple downstream tasks.

[0026] Step S1012: Match multiple corresponding LoRA modules according to the multiple downstream tasks.

[0027] Step S1013: Create a LoRA structure of multiple LoRA modules.

[0028] Step S1014: Initialize matrix parameters of the LoRA structure of the plurality of LoRA modules.

[0029] Furthermore, the LoRA-based large model task optimization and adjustment method also includes the following steps: Step S102: Acquire data sets of multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data.

[0030] In an embodiment of the present invention, by obtaining data sets of multiple downstream tasks, based on multiple data sets, keeping the main model parameters unchanged, training the corresponding LoRA modules, constructing loss functions corresponding to multiple LoRA modules and multiple downstream tasks, and then performing parameter training on multiple LoRA modules according to multiple loss functions, and recording parameter data, specifically, the loss function is: ; ; in, Indicates Downstream tasks, Indicates LoRA modules, Indicates Datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0031] Specifically, Figure 3 A flow chart of performing parameter training on multiple LoRA modules in the method provided in an embodiment of the present invention is shown.

[0032] Among them, in the preferred embodiment provided by the present invention, the acquisition of data sets of multiple downstream tasks, parameter training of multiple LoRA modules, and recording parameter data specifically include the following steps: Step S1021: Acquire data sets of multiple downstream tasks.

[0033] Step S1022: Based on the multiple data sets, construct loss functions corresponding to the multiple LoRA modules and the multiple downstream tasks.

[0034] Step S1023: Perform parameter training on the multiple LoRA modules according to the multiple loss functions, and record parameter data.

[0035] Furthermore, the LoRA-based large model task optimization and adjustment method also includes the following steps: Step S103: obtain the Prompts of multiple downstream tasks, use the sentence embedding model to map the multiple Prompts and corresponding LoRA modules to the same embedding space, and determine the embedding representations of the multiple LoRA modules.

[0036] In an embodiment of the present invention, by obtaining the prompts of multiple downstream tasks, the sentence embedding model is used to convert the prompts of each downstream task into an embedded representation, and the embedded representations of the samples of different specific fields corresponding to the multiple downstream tasks are determined, and then the multiple prompts and the corresponding LoRA modules are mapped to the same embedding space, and the embedded representations of the multiple LoRA modules are determined. Specifically, the embedded representations of the multiple LoRA modules are: ; in, Indicates The embedded representation of a LoRA module, Representation Task No. The embedding representation of samples in a specific domain, A sample of a specific area.

[0037] Step S104, calculating contribution weights between multiple LoRA modules and multiple downstream tasks, and calculating optimization and adjustment outputs according to the multiple contribution weights.

[0038] In an embodiment of the present invention, by calculating the cosine similarities between multiple LoRA modules and multiple downstream tasks, and then calculating the contribution weights of multiple LoRA modules based on the multiple cosine similarities, and then calculating the optimization adjustment output according to the multiple contribution weights, the most suitable current task and LoRA module are dynamically selected to achieve the optimization adjustment of the model. Specifically, the calculation formula of multiple cosine similarities is: ; in, Indicates The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; The calculation formula for multiple contribution weights is: ; in, Indicates The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula for the optimized adjustment output is: .

[0039] Specifically, Figure 4 A flow chart of calculating the optimization adjustment output in the method provided by an embodiment of the present invention is shown.

[0040] Among them, in the preferred embodiment provided by the present invention, the calculation of the contribution weights between multiple LoRA modules and multiple downstream tasks, and the calculation of the optimization adjustment output according to the multiple contribution weights specifically include the following steps: Step S1041, calculating the cosine similarity between multiple LoRA modules and multiple downstream tasks.

[0041] Step S1042: Calculate contribution weights of the multiple LoRA modules based on the multiple cosine similarities.

[0042] Step S1043: Calculate the optimized adjustment output according to the multiple contribution weights.

[0043] Furthermore, Figure 5 The application architecture diagram of the LoRA-based large model task optimization and adjustment system provided in an embodiment of the present invention is shown.

[0044] Among them, in another preferred embodiment provided by the present invention, a large model task optimization and adjustment system based on LoRA includes: The LoRA module creation unit 101 is used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules.

[0045] In an embodiment of the present invention, the large language model needs to adapt to a variety of different downstream tasks. Different downstream tasks have significant heterogeneity in many aspects. The LoRA module creation unit 101 determines multiple downstream tasks by monitoring the application tasks, and then matches multiple corresponding LoRA modules according to the multiple downstream tasks. By creating the LoRA structure of multiple LoRA modules, the matrix parameters of the multiple LoRA modules are initialized based on the LoRA structure. Specifically, the LoRA structure is composed of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, For the output dimension, initialize the matrix parameters of the LoRA structure of multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to a Gaussian distribution.

[0046] The LoRA parameter training unit 102 is used to obtain data sets of multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data.

[0047] In an embodiment of the present invention, the LoRA parameter training unit 102 obtains data sets of multiple downstream tasks, keeps the main model parameters unchanged based on multiple data sets, trains the corresponding LoRA modules, constructs loss functions corresponding to multiple LoRA modules and multiple downstream tasks, and then performs parameter training on multiple LoRA modules according to multiple loss functions, and records parameter data. Specifically, the loss function is: ; ; in, Indicates Downstream tasks, Indicates LoRA modules, Indicates Datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

[0048] The LoRA embedding analysis unit 103 is used to obtain the prompts of multiple downstream tasks, use the sentence embedding model, map the multiple prompts and corresponding LoRA modules to the same embedding space, and determine the embedding representations of the multiple LoRA modules.

[0049] In the embodiment of the present invention, the LoRA embedding analysis unit 103 obtains the prompts of multiple downstream tasks, and then uses the sentence embedding model to convert the prompts of each downstream task into an embedded representation, determines the embedded representations of the multiple downstream tasks corresponding to the samples in different specific fields, and then maps the multiple prompts and the corresponding LoRA modules to the same embedding space, and determines the embedded representations of the multiple LoRA modules. Specifically, the embedded representations of the multiple LoRA modules are: ; in, Indicates The embedded representation of a LoRA module, Indicates the task No. The embedding representation of domain-specific samples is A sample of a specific area.

[0050] The optimization output processing unit 104 is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimization adjustment output according to the multiple contribution weights.

[0051] In an embodiment of the present invention, the optimization output processing unit 104 calculates the cosine similarities between multiple LoRA modules and multiple downstream tasks, and then calculates the contribution weights of multiple LoRA modules based on the multiple cosine similarities, and then calculates the optimization adjustment output according to the multiple contribution weights, dynamically selects the most suitable current task and LoRA module, and realizes the optimization adjustment of the model. Specifically, the calculation formula of multiple cosine similarities is: ; in, Indicates The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; The calculation formula for multiple contribution weights is: ; in, Indicates The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula for the optimized adjustment output is: .

[0052] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0053] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0054] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A large model task optimization and adjustment method based on LoRA, characterized in that: The method specifically comprises the following steps: Perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules; Obtain data sets of multiple downstream tasks, perform parameter training on multiple LoRA modules, and record parameter data; Obtaining multiple prompts of the downstream tasks, using a sentence embedding model, mapping the multiple prompts and corresponding LoRA modules to the same embedding space, and determining the embedding representations of the multiple LoRA modules; Calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output based on the multiple contribution weights.

2. The large model task optimization and adjustment method based on LoRA according to claim 1 is characterized in that: The application task monitoring, determining multiple downstream tasks, matching multiple corresponding LoRA modules, and initializing the matrix parameters of multiple LoRA modules specifically include the following steps: Conduct application task monitoring and identify multiple downstream tasks; According to the plurality of downstream tasks, matching a plurality of corresponding LoRA modules; Creating a LoRA structure of multiple LoRA modules; Initialize the matrix parameters of the LoRA structure of the plurality of LoRA modules.

3. The large model task optimization and adjustment method based on LoRA according to claim 2 is characterized in that: The LoRA structure consists of two matrices: ,in, is the rank of the low-rank decomposition, is the input dimension, is the output dimension.

4. The LoRA-based large model task optimization and adjustment method according to claim 3 is characterized in that: In the matrix parameters of the LoRA structure of the initialization multiple LoRA modules, the matrix Initialized to an all-zero matrix, the matrix Initialized to a Gaussian distribution.

5. The large model task optimization and adjustment method based on LoRA according to claim 3 is characterized in that: The step of obtaining a plurality of data sets of the downstream tasks, performing parameter training on a plurality of the LoRA modules, and recording parameter data specifically comprises the following steps: Acquire data sets of multiple downstream tasks; Based on the multiple data sets, construct loss functions corresponding to the multiple LoRA modules and the multiple downstream tasks; According to the multiple loss functions, parameter training is performed on the multiple LoRA modules, and parameter data is recorded.

6. The LoRA-based large model task optimization and adjustment method according to claim 5 is characterized in that: The loss function is: ； ； in, Indicates Downstream tasks, Indicates LoRA modules, Indicates Datasets, is the input vector, It is with The output vector corresponding to the downstream task, is the true label, are the main model parameters and remain unchanged.

7. The LoRA-based large model task optimization and adjustment method according to claim 6 is characterized in that: The embedding of multiple LoRA modules is represented as: ； in, Indicates The embedded representation of a LoRA module, Indicates the task No. The embedding representation of samples in a specific domain, A sample of a specific area.

8. The LoRA-based large model task optimization and adjustment method according to claim 7 is characterized in that: The calculation of contribution weights between multiple LoRA modules and multiple downstream tasks, and the calculation of optimization and adjustment output according to the multiple contribution weights specifically include the following steps: Calculate the cosine similarity between multiple LoRA modules and multiple downstream tasks; Based on the multiple cosine similarities, calculating the contribution weights of the multiple LoRA modules; An optimized adjustment output is calculated based on the plurality of contribution weights.

9. The LoRA-based large model task optimization and adjustment method according to claim 8 is characterized in that: The calculation formula of the multiple cosine similarities is: ； in, Indicates The cosine similarity corresponding to the LoRA modules, For input The embedding representation of represents the vector dot product, represents the L2 norm; The calculation formula of the multiple contribution weights is: ； in, Indicates The contribution weights corresponding to the LoRA modules are LoRA modules; The calculation formula of the optimization adjustment output is: 。 10. A large model task optimization and adjustment system based on LoRA, characterized in that: The system includes a LoRA module creation unit, a LoRA parameter training unit, a LoRA embedding analysis unit and an optimization output processing unit, wherein: A LoRA module creation unit, used to perform application task monitoring, determine multiple downstream tasks, match multiple corresponding LoRA modules, and initialize matrix parameters of multiple LoRA modules; A LoRA parameter training unit, used to obtain a plurality of data sets of the downstream tasks, perform parameter training on the plurality of LoRA modules, and record parameter data; A LoRA embedding analysis unit, used to obtain prompts of multiple downstream tasks, map multiple prompts and corresponding LoRA modules to the same embedding space using a sentence embedding model, and determine the embedding representations of multiple LoRA modules; The optimized output processing unit is used to calculate the contribution weights between multiple LoRA modules and multiple downstream tasks, and calculate the optimized adjustment output according to the multiple contribution weights.

Images