Efficient deployment method and system for large model adapters across multiple domains and tasks
The method addresses the challenge of deploying large language models across multiple tasks and domains by training a selector using LORA fine-tuning and data filtering, resulting in efficient and accurate multi-task performance with reduced resource consumption.
Patent Information
- Application Number
- CN202411191319.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2044-08-28
AI Technical Summary
Large language models require significant resources for deployment and face challenges in efficient, stable deployment, especially in resource-constrained environments, and struggle with vertical domain adaptation and multi-task performance across different domains.
A method for training a selector using LORA fine-tuning and data filtering to deploy a single large model across multiple tasks and domains, reducing resource consumption and improving efficiency through adapter indexing and selective deployment strategies.
This approach enables efficient and accurate multi-task deployment of large language models across domains with reduced resource usage, improving performance and adaptability in resource-limited settings.
Smart Images
Figure CN119179495B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence and natural language processing, and relates to a method and system for efficiently deploying large model adapters across multiple domains and tasks, and specifically to a training method for a multi-adapter selector across multiple domains and tasks based on an efficient fine-tuning method for large model parameters, and a large model multi-adapter joint deployment system based on the selector. Background Art
[0002] With the rapid development of artificial intelligence technology and model training equipment and technology, large language models based on transformer architecture, such as the GPT series, Qwen series, and Llama series, have become the core driving force in the field of natural language processing. These models with a large number of parameters have learned rich language knowledge and intelligent context understanding and generation capabilities through continuous and sufficient pre-training on massive text data, and can demonstrate strong performance in various natural language processing tasks.
[0003] However, these models are often accompanied by huge parameter scales and computing requirements, which place extremely high demands on hardware resources. In practical applications, especially in resource-constrained environments, how to efficiently and stably deploy large language models has become a technical problem that needs to be solved urgently. At the same time, with the deepening of industry applications, the direct application of general large-scale language models to vertical fields faces many challenges, such as insufficient understanding of industry terminology, output that does not meet industry standards, and limited reasoning capabilities. Therefore, vertical field adaptation optimization and efficient and stable deployment of large language models have become an important direction for current technological development.
[0004] When applying LLMs to a specific domain or task, it is necessary to efficiently fine-tune the model parameters for the dataset of the task. This method inserts a small number of trainable parameters such as small neural network layers or low-rank matrices at different levels of the model, and only fine-tunes these newly added parameters, while keeping most of the parameters of the pre-trained model fixed. This method significantly reduces the number of parameters that need to be trained, speeds up the fine-tuning process, and reduces the risk of overfitting.
[0005] In traditional large-model applications, each task often requires efficient fine-tuning of parameters of the base model using domain-specific data, and then deploying this independent model equipped with a fine-tuning adapter in the video memory that meets the maximum context width of the model, and opening the model generation application program interface for user access. As the number of tasks increases, this one-to-one deployment mode not only consumes a lot of computing resources and time, but may also face the dilemma of insufficient video memory resources. For example, users continuously call three services in the medical field deployed using large models: information extraction, relationship extraction, and entity naming recognition to read cases, thereby preventing and pre-diagnosing diseases. However, due to resource constraints, these three services are often deployed in different servers, which increases deployment costs and resource consumption, increases user call delays, and often performs poorly in cross-domain deployment scenarios and requirements involving different tasks in different fields. However, the traditional training method of using multiple tasks simultaneously not only increases the consumption of model training but also reduces the performance of the model in a single task. The traditional multi-task deployment method is also not suitable for multi-task deployment and calling of large models.
[0006] Based on the above research background, it can be found that the deployment and application of large language models in vertical fields often finds it difficult to achieve effective deployment in multiple fields and multiple tasks and there is a problem of resource constraints. Summary of the invention
[0007] In order to solve the above problems existing in the prior art, the present invention provides a method and system for efficient deployment of large model adapters for cross-domain and multi-task. The present invention proposes a training method for an adapter selector based on the lora fine-tuning method and the data screening method, so that a single large language model equipped with this selector and multiple fine-tuning adapters can solve multiple tasks in the same field or tasks in different fields at the same time, significantly reducing resource consumption and improving the overall efficiency of the model.
[0008] The objective of the present invention is achieved through the following technical solutions:
[0009] An efficient deployment method for a large model adapter for cross-domain and multi-task tasks includes the following steps:
[0010] Step S1: Construct model adapter index:
[0011] Step S11: select a text training instruction dataset based on multiple fields and multiple tasks to be deployed, match each text training instruction dataset with the corresponding instruction fine-tuning prompt, and then use the lora fine-tuning method on the same basic large language model to perform instruction fine-tuning training, obtain the adapter weight corresponding to each text training instruction dataset and save it;
[0012] Step S12: construct a two-dimensional data index table based on domains and tasks. The index uses domain-task as a key and the file path of the adapter as a value. The corresponding adapter is searched using domain-task as an index to obtain an adapter index, a list of domains to be deployed, and a list of tasks to be deployed.
[0013] Step S2: construct a mixed representation dataset for training the selector:
[0014] For each text training instruction dataset, each piece of data is passed through the embedding layer structure of the basic model to obtain a low-dimensional embedding vector containing semantic features. The Kmeans clustering algorithm is used to screen the embedding vector set, and the set of cluster centers is used as the representative representation dataset of this text training instruction dataset. The representation datasets of different text training instruction datasets are mixed in disorder to obtain the selector training set. The specific steps are as follows:
[0015] Step S21, using a word embedding method based on the embedding layer of the basic model, calculate a low-dimensional embedding vector for each piece of data in each text training instruction data set to obtain an embedding vector set;
[0016] Step S22: cluster each text training instruction data set using the Kmeans clustering algorithm based on the output of the embedding layer of the basic model, select the number of cluster centers based on the minimum amount of training data, generally 500 or 10% of the minimum amount of data, and aggregate the original text data corresponding to each cluster center of each clustering to obtain the representation data set of this clustering;
[0017] Step S23, randomly mixing the representation data sets from different text training instruction data sets to obtain a selector training set;
[0018] Step S3, training selector:
[0019] Step S31, match each data of the selector training set obtained in step S2, the list of areas to be deployed and the list of tasks to be deployed obtained in step S1 with the general selector training prompt to obtain a final training set containing the prompt;
[0020] Step S32: Use the data set obtained in step S31 to perform instruction fine-tuning training on the basic large language model in step S1 using the lora fine-tuning method to obtain the selector weight and save it;
[0021] Step S4: Joint deployment of selector-based adapters:
[0022] Step S41, deployment strategy: determine the number of basic large language models to be deployed, the location and number of selectors, and the location of adapters based on computing resources;
[0023] Step S42, model loading: Apply the deployment strategy to obtain the deployment parameter configuration and adapter configuration file, load the basic large language model, selector, and adapters into the appropriate corresponding positions of the video memory resources, and load the adapter index, list of fields to be deployed, and list of tasks to be deployed obtained in step S1 into the appropriate positions of the memory resources;
[0024] Step S43, Adapter registration: register the selector and adapters in the basic large language model, establish a connection through the adapters index, and match each adapter with the adapter_name in the corresponding adapters index. The content of adapter_name is related to the domain and task corresponding to the adapter.
[0025] Step S44, model generation: After the question data matches the prompt of the selector, the model loaded with the selector is entered, and the domains and tasks related to the question are output; the corresponding adapter is searched in the adapters index according to the related domains and tasks; the model loaded with the selector is decoupled from the selector and reorganized with the corresponding adapter; after the question data matches the prompt of the domain task, the model loaded with the adapter is entered, and the output is the answer to the question;
[0026] Step S45: Expose the model generation process of step S44 as an API interface, that is, complete the deployment for convenient calling by users.
[0027] A system for implementing the above-mentioned cross-domain multi-task large model adapter efficient deployment method includes an adapter index construction module, a selector training data construction module, a selector training module, and an adapter joint deployment module, wherein:
[0028] The adapter index construction module is responsible for constructing a domain task adapter index through the trained adapter, so as to find and use the corresponding adapter using the domain-task;
[0029] The selector training data construction module is responsible for applying the data screening algorithm on the adapter training data and mixing it, and finally obtaining a mixed representation data set;
[0030] The selector training module is responsible for using a reasonable and effective model fine-tuning method and a mixed representation dataset, and applying the selector's dedicated prompt to train the model;
[0031] The adapter joint deployment module is responsible for combining models, selectors, and adapters, and completing the reasoning of the model equipped with the selector.
[0032] Compared with the prior art, the present invention has the following advantages:
[0033] 1. The present invention proposes an adapter training method based on adapter selection. This method can complete the selection of the adapter of the model according to the field and task with training under a small amount of data, saving training resources for multi-task training of the model, improving the accuracy of multi-task reasoning of the model, and supporting separate training and unified deployment of multiple tasks, avoiding the difficult construction of multi-task training data.
[0034] 2. The present invention proposes a deployment method based on an adapter selector, which can deploy a large language model adapted for multiple fields and multiple tasks under limited video memory resources, saving deployment resources and realizing effective multi-task deployment of large models in vertical fields.
[0035] 3. The present invention proposes a deployment strategy based on a given video memory. According to the given resources, the deployment resources are used as much as possible to load the model and it is ensured that there will be no shortage of video memory, thereby improving the video memory utilization rate of the deployment resources.
[0036] 4. The present invention proposes the concept of adapter selector in the deployment of large language models, which can accurately determine the field and task corresponding to the input based on the input of the model, thereby guiding the system to accurately match and reorganize the large model with the model adapter of this field and task, thereby improving the accuracy of multi-task reasoning; at the same time, a selector training method and an inference process and deployment method of a large model carrying a selector are proposed, thereby improving the resource utilization of limited resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A framework diagram for efficiently deploying large model adapters for cross-domain and multi-task tasks.
[0038] Figure 2 Flowchart of an efficient deployment method for large model adapters targeting cross-domain multi-tasks.
[0039] Figure 3 Flowchart for training adapter index and adapter selector.
[0040] Figure 4Deployment model structure diagram for adapter union. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.
[0042] The present invention provides an efficient deployment method for large model adapters across multiple domains and tasks, such as Figure 2 As shown, the method comprises the following steps:
[0043] Step S1: Construct model adapter index:
[0044] This step mainly selects text training instruction datasets based on multiple fields and multiple tasks to be deployed, such as financial, medical, legal and other datasets. For each dataset, it is matched and compared with the corresponding domain-based instruction fine-tuning prompt, and then the lora fine-tuning method is used on a basic large language model with excellent performance and appropriate parameter quantity to perform instruction fine-tuning training to obtain and save the adapter weights corresponding to each dataset. Generally, when the parameter quantity of the basic model is 7B, the size of a trained fine-tuning adapter is 200-300MB. Then, a two-dimensional data index table is constructed based on the domain and task. The index uses the domain-task as the key and the file path of the adapter as the value. The corresponding adapter can be searched with the domain-task as the index. This step can obtain the adapter index required by the system, the list of domains to be deployed, and the list of tasks to be deployed. Specific examples are as follows: Figure 4 shown.
[0045] Step S2: construct a mixed representation dataset for training the selector:
[0046] In this step, for each text training data set in step S1, each data is passed through the embedding layer structure of the basic model to obtain a low-dimensional embedding vector containing semantic features, and the embedded vector set is screened using the Kmeans clustering algorithm based on the distance metric. The present invention uses the output of the embedding layer of the basic model, takes the average of the output word vector as the representation vector of the sentence, and uses cosine similarity as the distance metric of the Kmeans method, and selects the number of cluster centers based on the minimum number of training data, which is generally 500 or 10% of the minimum data volume. And the set of cluster centers of the clusters is used as the representative representation data set of this training data set. The representation data sets of different data sets are mixed in disorder to obtain the selector training set. Specific examples are as follows Figure 4 shown.
[0047] Step S3, training selector:
[0048] Match each data of the selector dataset, the list of fields to be deployed obtained in step S1, and the list of tasks to be deployed together with the general selector training prompt to obtain the final training set containing the prompt. Use the lora fine-tuning method on the basic large language model in step S1 to perform instruction fine-tuning training using this dataset to obtain the selector weights and save them. The following parameter configurations can be used for the lora fine-tuning method: rank is 4 or 8, pruning rate is 0.95, and contribution is 16 or 32; at the same time, advanced lora methods such as rslora and Pissa can be used, and the training rounds are not less than three rounds; after training, the selector can be tested with randomly extracted data.
[0049] Step S4: Joint deployment of selector-based adapters:
[0050] The selector-based adapter joint deployment system includes the following deployment strategies and processes:
[0051] Apply the deployment strategy to obtain the deployment parameter configuration and adapter configuration file. The deployment strategy determines the number of basic large models to be deployed, the location and number of selectors, and the location of adapters based on computing resources. The available deployment strategies are as follows:
[0052] According to the given deployment resources, load all selectors and adapters, and measure the memory occupied as the total adapter memory occupation, and the remaining memory is the memory occupation of the base model; load the model into the memory, use the maximum context length (token number) supported by the model to form the input of the composition model, and measure the memory occupied at this time as the memory occupied by a single model; according to the available memory of the base model and the memory occupied by the inference of a single model, get the maximum number of base models that the deployment resources can accommodate; load the above number of base models, and allocate adapters and selectors to these base models, and the allocation meets the following strategy:
[0053] 1. There must be a selector mounted on a base model;
[0054] 2. Adapters in the same field should be assigned to one base model as much as possible;
[0055] 3. If there is still space left, you can load the selector.
[0056] Through the deployment parameter configuration and adapter configuration file, the basic large language model, the selector obtained in step S3, and the adapters are loaded into the appropriate corresponding positions of the video memory resources, and the adapter index, the list of fields to be deployed, and the list of tasks to be deployed obtained in step S1 are loaded into the appropriate positions of the memory resources. Then, the adapter is registered, that is, the selector and adapters are registered in the large model, and the connection is established through the adapters index, and then the model API can be deployed.
[0057] The model generation process is as follows Figure 3 As shown in the figure: after the problem data, i.e. the INPUT of the model, matches the prompt of the selector, it enters the model loaded with the selector, and the output is the problem-related fields and tasks, i.e. finance and task extraction; according to the relevant fields and tasks, the corresponding adapter is found in the adapters index, and the model loaded with the selector is decoupled from the selector and reorganized with the corresponding adapter; after the problem data matches the prompt of the domain task, it enters the model loaded with the adapter, and the task of extracting important information from the INPUT can be completed. The combination of the adapter output is the answer to the question.
[0058] The present invention also provides a system for implementing the above-mentioned large model adapter efficient deployment method for cross-domain multi-task, such as Figure 1 As shown, the system includes an adapter index construction module, a selector training data construction module, a selector training module, and an adapter joint deployment module, wherein:
[0059] The adapter index construction module is responsible for constructing a domain task adapter index through the trained adapter, so as to find and use the corresponding adapter using the domain-task;
[0060] The selector training data construction module is responsible for applying the data screening algorithm on the adapter training data and mixing it, and finally obtaining a mixed representation data set;
[0061] The selector training module is responsible for using a reasonable and effective model fine-tuning method and a mixed representation dataset, and applying the selector's dedicated prompt to train the model;
[0062] The adapter joint deployment module is responsible for combining models, selectors, and adapters, and completing the reasoning of the model equipped with the selector.
Claims
1. An efficient deployment method for large model adapters across multiple domains and tasks, characterized by The method comprises the following steps: Step S1: Construct model adapter index: Step S11: select a text training instruction dataset based on multiple fields and multiple tasks to be deployed, match each text training instruction dataset with the corresponding instruction fine-tuning prompt, and then use the lora fine-tuning method on the same basic large language model to perform instruction fine-tuning training, obtain the adapter weight corresponding to each text training instruction dataset and save it; Step S12: construct a two-dimensional data index table based on domains and tasks. The index table uses domain-task as a key and the file path of the adapter as a value. The corresponding adapter is searched using domain-task as an index to obtain an adapter index, a list of domains to be deployed, and a list of tasks to be deployed. Step S2: construct a mixed representation dataset for training the selector: For each text training instruction dataset, each piece of data is passed through the embedding layer structure of the basic model to obtain a low-dimensional embedding vector containing semantic features. The Kmeans clustering algorithm is used to screen the embedding vector set, and the set of cluster centers is used as the representative representation dataset of this text training instruction dataset. The representation datasets of different text training instruction datasets are mixed in disorder to obtain the selector training set. Step S3, training selector: Step S31, match each data of the selector training set obtained in step S2, the list of areas to be deployed and the list of tasks to be deployed obtained in step S1 with the general selector training prompt to obtain a final training set containing the prompt; Step S32: Use the data set obtained in step S31 to perform instruction fine-tuning training on the basic large language model in step S1 using the lora fine-tuning method to obtain the selector weight and save it; Step S4: Joint deployment of selector-based adapters: Step S41, deployment strategy: determine the number of basic large language models to be deployed, the location and number of selectors, and the location of adapters based on computing resources; Step S42, model loading: Apply the deployment strategy to obtain the deployment parameter configuration and adapter configuration file, load the basic large language model, selector, and adapters into the appropriate corresponding positions of the video memory resources, and load the adapter index, list of fields to be deployed, and list of tasks to be deployed obtained in step S1 into the appropriate positions of the memory resources; Step S43, Adapter registration: register the selector and adapters in the basic large language model, establish a connection through the adapters index, and match each adapter with the adapter_name in the corresponding adapters index. The content of adapter_name is related to the domain and task corresponding to the adapter. Step S44, model generation: After the question data matches the prompt of the selector, the model loaded with the selector is entered, and the domains and tasks related to the question are output; the corresponding adapter is searched in the adapters index according to the related domains and tasks; the model loaded with the selector is decoupled from the selector and reorganized with the corresponding adapter; after the question data matches the prompt of the domain task, the model loaded with the adapter is entered, and the output is the answer to the question; Step S45: Expose the model generation process of step S44 as an API interface, that is, complete the deployment for convenient calling by users.
2. The efficient deployment method of large model adapters for cross-domain and multi-task according to claim 1 is characterized in that The specific steps of step S2 are as follows: Step S21, using a word embedding method based on the embedding layer of the basic model, calculate a low-dimensional embedding vector for each piece of data in each text training instruction data set to obtain an embedding vector set; Step S22: cluster each text training instruction data set using the Kmeans clustering algorithm based on the output of the embedding layer of the basic model, select the number of cluster centers based on the minimum amount of training data, which is 500 or 10% of the minimum amount of data, and aggregate the original text data corresponding to each cluster center of each clustering to obtain the representation data set of this clustering; Step S23: randomly mix the representation data sets from different text training instruction data sets to obtain a selector training set.
3. A system for implementing the efficient deployment method of a large model adapter for cross-domain and multi-task as described in any one of claims 1-2, characterized in that The method includes an adapter index construction module, a selector training data construction module, a selector training module, and an adapter joint deployment module, wherein: The adapter index construction module is responsible for constructing a domain task adapter index through the trained adapter, so as to find and use the corresponding adapter using the domain-task; The selector training data construction module is responsible for applying the data screening algorithm on the adapter training data and mixing it, and finally obtaining a mixed representation data set; The selector training module is responsible for using a reasonable and effective model fine-tuning method and a mixed representation dataset, and applying the selector's dedicated prompt to train the model; The adapter joint deployment module is responsible for combining models, selectors, and adapters, and completing the reasoning of the model equipped with the selector.