A large language model quantization federated fine-tuning method and system for heterogeneous tasks

Abstract

The application provides a large language model quantization federated fine-tuning method and system for heterogeneous tasks, which comprises the following steps: obtaining available resources and training data sets corresponding to each of a plurality of clients; determining local matrices and quantization levels corresponding to each of the plurality of clients based on the available resources corresponding to each of the plurality of clients; determining large language models corresponding to each of the plurality of clients based on the quantization levels and the local matrices corresponding to each of the plurality of clients; training the large language models of the plurality of clients respectively based on the training data sets corresponding to each of the plurality of clients to obtain model parameters corresponding to each of the plurality of clients; and performing global aggregation of the model parameters to determine global model parameters based on the model parameters corresponding to each of the plurality of clients; and performing quantization federated fine-tuning based on the global model parameters. The above scheme can improve the generalization performance of the model while reducing the time occupied by the model training.

Description

Technical Field

[0001] This application relates to the fields of machine learning and edge computing technology, specifically to a method and system for quantized federated fine-tuning of large language models for heterogeneous tasks. Background Technology

[0002] Large language models have become the cornerstone of modern natural language processing systems, and their full potential typically requires fine-tuning with domain-specific or private datasets to be realized. However, increasingly stringent data privacy and security regulations have made it difficult or even impossible to centrally collect sensitive data for large language model fine-tuning. Fortunately, federated learning offers a promising alternative, allowing multiple different clients to collaboratively train models without sharing local data, thus protecting privacy while utilizing distributed data resources. Although federated learning opens new avenues for privacy-preserving large language model fine-tuning, its practical application still faces two major challenges: First, the massive parameter scale of large language models places stringent demands on the memory and computing power of client devices, making it difficult for many edge devices to complete full model training. Second, client devices in federated systems are typically equipped with heterogeneous hardware resources (such as GPU memory) and diverse local data distributions. This heterogeneity can easily lead to the slacker problem during training, potentially affecting the global generalization performance of the aggregated model. Summary of the Invention

[0003] This application aims to provide a method and system for federated fine-tuning of large language models for heterogeneous tasks, which can improve the generalization performance of the model while reducing the time spent on model training.

[0004] The technical solution of this application is implemented as follows: In a first aspect, embodiments of this application provide a method for quantized federated fine-tuning of large language models for heterogeneous tasks, the method comprising: Obtain the available resources and training datasets for each of the multiple clients; Based on the available resources corresponding to each of the multiple clients, determine the local matrix and quantization level corresponding to each of the multiple clients; Based on the quantization level and local matrix corresponding to each of the multiple clients, the large language model corresponding to each of the multiple clients is determined; and the large language model of each of the multiple clients is trained using the training dataset corresponding to each of the multiple clients to obtain the model parameters corresponding to each of the multiple clients. Based on the model parameters corresponding to each of the multiple clients, global parameter aggregation is performed to determine global model parameters; so as to enable quantitative federated fine-tuning based on the global model parameters.

[0005] In the above scheme, determining the local matrix and quantization level corresponding to each of the multiple clients based on their respective available resources includes: Based on the available resources corresponding to each of the multiple clients, feature analysis is performed to determine multiple candidate ranks corresponding to each of the multiple clients; For each of the plurality of clients, calculate the utility function corresponding to each of the plurality of candidate ranks for each client; Based on the utility functions corresponding to the multiple candidate ranks of each client, maximum rank screening is performed to determine the local matrix and quantization level corresponding to each of the multiple clients.

[0006] In the above scheme, the available resources include memory resources and computing resources; The step of performing feature analysis based on the available resources corresponding to each of the multiple clients to determine multiple candidate ranks corresponding to each of the multiple clients includes: Based on the memory resources corresponding to each of the multiple clients, feature analysis is performed to determine the memory characteristics corresponding to each of the multiple clients; wherein, the memory characteristics represent the free memory capacity of the client; Based on the computing resources corresponding to each of the multiple clients, feature analysis is performed to determine the computing characteristics corresponding to each of the multiple clients; wherein, the computing characteristics characterize the computing speed of the client; Based on the memory characteristics and computational characteristics of each of the multiple clients, resource analysis is performed to determine multiple candidate ranks for each of the multiple clients.

[0007] In the above scheme, calculating the utility function corresponding to each of the multiple candidate ranks for each of the multiple clients includes: For each of the multiple clients, historical data, client weighting factors, and resource costs are obtained; wherein, the resource cost representation is modeled as a linear function of different ranks; Based on the historical data, forward propagation estimation is performed for different rank configurations to determine the performance gain; Based on the client weight factor, the performance gain, and the resource cost, the utility function corresponding to each of the multiple candidate ranks for each client is determined.

[0008] In the above scheme, the step of performing maximum rank screening based on the utility functions corresponding to the multiple candidate ranks of each client to determine the local matrix and quantization level corresponding to each of the multiple clients includes: Determine the memory capacity corresponding to each of the multiple candidate ranks of each client and the number of floating-point operations corresponding to each of the multiple candidate ranks of each client; When the memory capacity is less than the memory resources and the number of floating-point operations is less than the computing resources, the maximum rank of each client is determined by filtering based on the utility functions corresponding to the multiple candidate ranks of each client. Based on the maximum rank of each client, a low-rank matrix is ​​configured to determine the local matrix corresponding to each of the multiple clients. Based on the local matrices corresponding to each of the multiple clients, the quantization level corresponding to the local matrix is ​​determined.

[0009] In the above scheme, determining the large language model corresponding to each of the multiple clients based on the quantization level and the local matrix includes: Determine the large language model weights corresponding to each of the multiple clients; and optimize the local matrix based on the large language model weights corresponding to each of the multiple clients to determine the local model corresponding to each of the multiple clients. Based on the quantization levels corresponding to each of the multiple clients, the local model is quantized to determine the large language model.

[0010] In the above scheme, the step of performing global parameter aggregation based on the model parameters corresponding to each of the multiple clients to determine the global model parameters includes: Based on the model parameters corresponding to each of the multiple clients, the low-rank adaptations of different ranks are aggregated and updated using the singular value decomposition method to obtain the updated model parameters corresponding to each of the multiple clients. Based on the updated model parameters corresponding to each of the multiple clients, model training is performed until the number of training iterations reaches a preset number, and the optimal model parameters corresponding to each of the multiple clients are obtained. The global model parameters are determined by performing global aggregation on the optimal model parameters corresponding to each of the multiple clients.

[0011] Secondly, embodiments of this application provide a large language model quantization federated fine-tuning system for heterogeneous tasks. The system includes: an acquisition module, a determination module, and a training module. The acquisition module is used to acquire the available resources and training datasets corresponding to each of the multiple clients; The determining module is used to determine the local matrix and quantization level corresponding to each of the multiple clients based on the available resources corresponding to each of the multiple clients. The training module is used to determine the large language model corresponding to each of the multiple clients based on the quantization level and the local matrix corresponding to each of the multiple clients; and to train the large language model of each of the multiple clients respectively using the training dataset corresponding to each of the multiple clients to obtain the model parameters corresponding to each of the multiple clients. The determining module is further configured to perform global parameter aggregation based on the model parameters corresponding to each of the multiple clients, and determine global model parameters, so as to enable quantitative federated fine-tuning based on the global model parameters.

[0012] Thirdly, embodiments of this application provide a large language model quantization federated fine-tuning device for heterogeneous tasks, comprising: a processor and a memory; wherein, The memory is used to store computer programs; The processor is configured to call and run the computer program from the memory to perform the method as described in the first aspect.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable instructions for causing a processor to perform the method described in the first aspect.

[0014] This application provides a method and system for federated fine-tuning of quantization for large language models in heterogeneous tasks. The method includes: acquiring available resources and training datasets for each of multiple clients; determining local matrices and quantization levels for each of the multiple clients based on their available resources; determining large language models for each of the multiple clients based on their quantization levels and local matrices; training the large language models for each of the multiple clients using their respective training datasets to obtain model parameters for each client; and performing global parameter aggregation based on the model parameters for each client to determine global model parameters; thereby enabling federated fine-tuning of quantization based on the global model parameters. In this scheme, rank allocation and model quantization are performed according to the available resources of the clients. Higher ranks are allocated to clients with processing capabilities to capture more complex local patterns and enhance global generalization ability, while lower ranks are allocated to resource-constrained clients to reduce their computational and communication load, achieving an efficient balance between model generalization ability, training latency, and memory usage. A heterogeneous quantization strategy is adopted to quantize the base model, allowing the client to load rank versions of different precisions based on available memory, further reducing the memory threshold and improving the model's generalization performance while reducing the time required for model training. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0016] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0017] Figure 1 A schematic diagram of an optional flowchart for a large language model quantization federated fine-tuning method for heterogeneous tasks provided in this application embodiment; Figure 2 A schematic diagram of the structure of a large language model quantization federated fine-tuning system for heterogeneous tasks provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a large language model quantization federated fine-tuning device for heterogeneous tasks, provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0019] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0020] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0021] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0022] Based on this, embodiments of this application provide a method for quantized federated fine-tuning of large language models for heterogeneous tasks. Figure 1 This is an optional flowchart illustrating an embodiment of a large language model quantization federated fine-tuning method for heterogeneous tasks provided in this application, which will combine... Figure 1 The steps shown are explained.

[0023] S101. Obtain the available resources and training datasets for each of the multiple clients.

[0024] In some embodiments of this application, available resources include memory resources and computing resources.

[0025] In some embodiments of this application, a large language model quantization federated fine-tuning method for heterogeneous tasks is adapted to the scenario of large language model quantization federated fine-tuning.

[0026] In some embodiments of this application, a large language model quantization federated fine-tuning method for heterogeneous tasks is adapted to a large language model quantization federated fine-tuning system for heterogeneous tasks.

[0027] In some embodiments of this application, available resources and training datasets corresponding to multiple clients are obtained; wherein, the training dataset is used to train the large language model corresponding to each of the multiple clients; and the available resources are used to determine the maximum rank corresponding to each of the multiple clients.

[0028] For example, memory resources can be memory capacity. Computing resources can be computing speed. .

[0029] S102. Based on the available resources of each client, determine the local matrix and quantization level of each client.

[0030] In some embodiments of this application, feature analysis is performed based on the available resources corresponding to each of the multiple clients to determine multiple candidate ranks corresponding to each client; for each of the multiple clients, the utility function corresponding to each of the multiple candidate ranks of each client is calculated; based on the utility function corresponding to each of the multiple candidate ranks of each client, maximum rank screening is performed to determine the local matrix and quantization level corresponding to each of the multiple clients.

[0031] S103. Based on the quantization level and local matrix of each client, determine the large language model corresponding to each client; and train the large language model of each client using the training dataset corresponding to each client to obtain the model parameters corresponding to each client.

[0032] In some embodiments of this application, the weights of the large language model corresponding to each of the multiple clients are determined; and the local matrix is ​​optimized based on the weights of the large language model corresponding to each of the multiple clients to determine the local model corresponding to each of the multiple clients; the local model is quantized based on the quantization level corresponding to each of the multiple clients to determine the large language model. The large language model for each of the multiple clients is trained using the training datasets corresponding to each of the multiple clients, thereby obtaining the model parameters corresponding to each of the multiple clients.

[0033] S104. Based on the model parameters corresponding to each of the multiple clients, perform global parameter aggregation to determine the global model parameters; so as to enable quantitative federated fine-tuning based on the global model parameters.

[0034] In some embodiments of this application, based on the model parameters corresponding to each of multiple clients, the singular value decomposition method is used to aggregate and update the low-rank adaptations of different ranks to obtain the updated model parameters corresponding to each of the multiple clients; based on the updated model parameters corresponding to each of the multiple clients, model training is performed until the number of training iterations reaches a preset number, and the optimal model parameters corresponding to each of the multiple clients are obtained; global aggregation of parameters is performed on the optimal model parameters corresponding to each of the multiple clients to determine the global model parameters; so as to enable quantitative federated fine-tuning based on the global model parameters.

[0035] For example, 1. The server initializes the model and sends the LoRA (Lower-Rank Adaptation) corresponding to the large language model to all clients. Each client selects the quantization level according to its own capabilities. 2. Client It will analyze its available resources (i.e., idle GPU memory, CPU utilization) and for each candidate rank Estimating utility 3. Client Choose the rank that maximizes utility while satisfying the constraints. Then instantiate its local LoRA matrix. and The client trains the model using a local dataset. A dynamically selected rank mechanism allows high-performance clients to capture more complex patterns, while constrained clients focus on efficiently learning the most critical core adaptive features. 4. After training, the client can record the relationship between the selected rank, the final local loss, and the actual training time. This data optimizes utility evaluation in subsequent rounds, enabling online adaptation to changing conditions (such as device overheating or background processes). The updated model parameters are then uploaded to the server. 5. Upon receiving the model parameters from the client, the server uses singular value decomposition to aggregate and update the LoRA of different ranks, and then sends the updated model parameters back to the client for the next round of training. This process is repeated until the model converges or reaches a pre-set number of rounds, determining the global model parameters; quantized federated fine-tuning is then implemented based on the global model parameters.

[0036] Understandably, rank allocation and model quantization are performed based on the available resources of the clients. Higher ranks are allocated to clients with processing power to capture more complex local patterns and enhance global generalization ability, while lower ranks are allocated to resource-constrained clients to reduce their computational and communication load. This achieves an efficient balance between model generalization ability, training latency, and memory usage. A heterogeneous quantization strategy is employed to quantize the base model, allowing clients to load rank versions of different precisions based on available memory, further lowering the memory threshold and reducing model training time while improving model generalization performance.

[0037] In some embodiments of this application, S102 can be implemented by S201-S203, as follows: S201. Based on the available resources corresponding to each of the multiple clients, perform feature analysis to determine multiple candidate ranks corresponding to each of the multiple clients.

[0038] In some embodiments of this application, feature analysis is performed based on the memory resources corresponding to each of the multiple clients to determine the memory features corresponding to each of the multiple clients; wherein, the memory features represent the free memory capacity of the client; feature analysis is performed based on the computing resources corresponding to each of the multiple clients to determine the computing features corresponding to each of the multiple clients; wherein, the computing features represent the computing speed of the client; and resource analysis is performed based on the memory features and computing features corresponding to each of the multiple clients to determine multiple candidate ranks corresponding to each of the multiple clients.

[0039] S202. For each of the multiple clients, calculate the utility function corresponding to each of the multiple candidate ranks for each client.

[0040] In some embodiments of this application, for each of the multiple clients, historical data, client weighting factors, and resource costs are obtained; wherein, the resource cost is represented as a linear function of different ranks; based on the historical data, forward propagation estimation of different rank configurations is performed to determine the performance gain; based on the client weighting factors, performance gains, and resource costs, the utility functions corresponding to the multiple candidate ranks of each client are determined.

[0041] S203. Based on the utility functions corresponding to the multiple candidate ranks of each client, perform maximum rank screening to determine the local matrix and quantization level corresponding to each client.

[0042] In some embodiments of this application, the memory capacity corresponding to each of the multiple candidate ranks of each client and the number of floating-point operations corresponding to each of the multiple candidate ranks of each client are determined; when the memory capacity is less than the memory resource and the number of floating-point operations is less than the computing resource, the maximum rank is determined by screening based on the utility function corresponding to each of the multiple candidate ranks of each client; based on the maximum rank of each client, the low-rank matrix is ​​configured to determine the local matrix corresponding to each of the multiple clients; based on the local matrix corresponding to each of the multiple clients, the quantization level corresponding to the local matrix is ​​determined.

[0043] For example, let's take a common application scenario of federated learning as an example: a base station and Each edge device has its own local dataset. ,in, .set up For the weights of the pre-trained large language model, the loss function , indicating equipment Relative to model weights The local training loss, of which, It is a predefined loss function. In low-rank adaptive federated learning, the loss function is determined by adjusting the low-rank matrix. , Optimize to make the local model The global objective is: It should be noted that, That is, the local matrix.

[0044] Client resources are heterogeneous, including memory capacity. Calculation speed and communication bandwidth Each client can choose its own LoRA tier. and quantification level (e.g., 4-bit, 8-bit, or 16-bit). This heterogeneity is addressed by integrating three core mechanisms: Heterogeneous quantization technology: The device applies client-specific quantization functions to the weights of the base model. The basic model will be used in a way that is suitable for the client. The quantization level mapping quantization function generates ,in, For quantization function, Basic model, This serves as the base model after quantization. This technique reduces memory usage and computational costs while maintaining data fidelity on high-precision devices. However, considering the differences in computing power and memory capacity among client devices, forcing a uniform quantization scheme is impractical. Devices equipped with more powerful GPUs can handle higher precision for better fidelity, while resource-constrained devices must use lower precision to participate. Therefore, a heterogeneous quantization scheme is introduced, allowing each client... Select appropriate quantization configuration based on its local hardware characteristics. .

[0045] set up Represents a set of supported quantization levels, where, Provides the highest accuracy ( ), Offers high accuracy (fp16). Provides the highest compression ratio (NF4). During the initialization phase, the central server distributes the full-precision base model weights to all clients. Then each client Apply its selected quantizer independently: in, The choice can be determined by a lightweight device analyzer that takes into account available GPU memory, peak performance, and power consumption limitations.

[0046] In the client During local training, the forward propagation process for the modified LoRA layer is as follows: in, It is the corresponding quantization base model for the client. and This is the corresponding low-rank adapter, using the same quantization level as the base model. The key point is that the gradient is only for... and To perform calculations and applications. (From...) The quantization error exists as a fixed, client-specific noise. This design ensures that all clients can collaboratively learn a shared low-dimensional update space, even though they are dealing with different representations of the base model. Unlike existing methods, which assume each client has fine-tuning capabilities during large model fine-tuning, this approach is overly idealistic. For example, fine-tuning a 16-bit LLaMA-7B using LoRA with a batch size of 1 requires approximately 19.5 GB of GPU memory. This makes loading and processing the model difficult for edge devices with limited memory and computing power, thus limiting the feasibility of federated fine-tuning of large language models and increasing training latency. Therefore, to reduce the memory overhead when fine-tuning large language models using LoRA, this paper proposes quantizing the pre-trained base model to minimize memory usage during the local training phase in federated learning.

[0047] Adaptive LoRA rank selection mechanism: Each client dynamically selects the optimal rank based on real-time resource analysis (including GPU memory and computing resources). This mechanism ensures that resource-rich devices can train high-capacity adapters, while resource-constrained devices focus on updating the base model.

[0048] Besides quantization, LoRA optimization (i.e., rank selection) can also vary depending on the device. Resource-rich devices can train higher-rank adapters to capture more complex task-specific features, while resource-poor devices use lower ranks to improve efficiency. To address this, a client-driven adaptive rank selection mechanism is proposed. This mechanism enables each client device to autonomously determine its optimal LoRA rank based on real-time resource configuration, local data characteristics, and performance feedback. This shift from a static server-controlled strategy to a dynamic client-side decision-making process is crucial for handling the inherent, time-varying system heterogeneity in federated networks. Traditional methods involve the server statically / dynamically determining the rank for each client: statically assigning LoRA ranks to clients cannot be evaluated in real-time based on client state, potentially causing interruptions in fine-tuning; while dynamically assigning ranks to clients overcomes the problems of static LoRA rank assignment, it may limit the contribution of capable clients to model fine-tuning. (A larger LoRA rank generally indicates better generalization performance. Here, we are suggesting that a client can use a LoRA rank of 256 to participate in fine-tuning, but the maximum LoRA rank assigned by the server is 16, thus limiting the client's contribution to model performance improvement.) Therefore, this paper proposes allowing the client itself to determine its rank to improve model performance.

[0049] After receiving the global model, each client A lightweight rank analyzer will be run before local training. This analyzer evaluates an objective function that balances two competing objectives: (1) Model capacity: predictive performance, approximated by the loss reduction on a small validation subset resulting from higher rank; (2) Resource cost: computation and memory usage of training and communication LoRA parameters, which varies with rank. It exhibits linear growth.

[0050] Formally, at the During the training round, the client Its rank is selected by solving the following optimization problem. : in, It is a proposed utility function. Represents rank It is from a candidate set Selected from Represents all possible rank configurations. and It is a client Available memory and computing resources They are different ranks Memory size occupied Representing different ranks The number of floating-point operations is a metric for measuring computational complexity, often used to evaluate the computational cost during inference or training. The corresponding utility function is expressed as follows: in, Represents the client-side weighting factor and performance gain. Resource costs can be estimated by performing several forward propagations on the retained data segments using different rank configurations. Modeled as A linear function.

[0051] The adaptive process for each round of communication is as follows: Client Perform feature analysis on available resources (i.e., idle GPU memory, CPU utilization) and for each candidate rank Estimating the utility function .

[0052] The client chooses the rank that maximizes utility under its constraints. Then instantiate its local LoRA matrix. and .

[0053] The client trains these matrices on its local dataset. Dynamically selected rank enables powerful clients to capture more complex patterns, while constrained clients focus on efficiently learning the most critical adaptive features.

[0054] After training, the client can record the correlation data between the selected rank, the final local loss, and the actual training time. This data will optimize the utility estimation in subsequent rounds, thereby enabling online adaptation to changing conditions (such as device overheating and background processes).

[0055] Global aggregation of heterogeneous hierarchical data: The server employs an aggregation scheme based on singular value decomposition to coordinate updates from different rank devices. This scheme avoids the inefficiency of zero-padding and compresses updates into a unified global model.

[0056] The embodiments of this application have the following beneficial effects: This application addresses the aggregation problem caused by client heterogeneity, task heterogeneity among different clients, and different ranks among different clients. It proposes an adaptive federated fine-tuning framework specifically designed for large language models, effectively addressing the dual challenges posed by system and task heterogeneity. Furthermore, by introducing client-aware dynamic LoRA rank allocation and heterogeneous quantization of the base model, it efficiently leverages the unique advantages of different clients. Additionally, a global aggregation mechanism based on singular value decomposition effectively coordinates the updates of different ranks. Breakthrough progress has been achieved in significantly improving generalization performance, shortening training time, and reducing memory consumption.

[0057] Based on the above embodiments, this application also provides a large language model quantization federated fine-tuning system for heterogeneous tasks, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a large language model quantization federated fine-tuning system for heterogeneous tasks provided in an embodiment of this application. The system includes: an acquisition module 201, a determination module 202, and a training module 203. The acquisition module 201 is used to acquire the available resources and training datasets corresponding to each of the multiple clients. The determining module 202 is used to determine the local matrix and quantization level corresponding to each of the multiple clients based on the available resources corresponding to each of the multiple clients. The training module 203 is used to determine the large language model corresponding to each of the multiple clients based on the quantization level and the local matrix corresponding to each of the multiple clients; and to train the large language model of each of the multiple clients respectively using the training dataset corresponding to each of the multiple clients to obtain the model parameters corresponding to each of the multiple clients. The determining module 202 is further configured to perform global parameter aggregation based on the model parameters corresponding to each of the multiple clients, and determine global model parameters, so as to enable quantitative federated fine-tuning based on the global model parameters.

[0058] In some embodiments of this application, the determining module 202 is further configured to perform feature analysis based on the available resources corresponding to each of the plurality of clients, and determine the plurality of candidate ranks corresponding to each of the plurality of clients; for each of the plurality of clients, calculate the utility function corresponding to the plurality of candidate ranks of each client; and perform maximum rank screening based on the utility function corresponding to the plurality of candidate ranks of each client, and determine the local matrix and quantization level corresponding to each of the plurality of clients.

[0059] In some embodiments of this application, the available resources include memory resources and computing resources; The determining module 202 is further configured to perform feature analysis based on the memory resources corresponding to each of the plurality of clients to determine the memory features corresponding to each of the plurality of clients; wherein the memory features represent the free memory capacity of the client; perform feature analysis based on the computing resources corresponding to each of the plurality of clients to determine the computing features corresponding to each of the plurality of clients; wherein the computing features represent the computing speed of the client; and perform resource analysis based on the memory features and computing features corresponding to each of the plurality of clients to determine multiple candidate ranks corresponding to each of the plurality of clients.

[0060] In some embodiments of this application, the determining module 202 is further configured to acquire historical data, client weight factors, and resource costs for each of the plurality of clients; wherein the resource cost is modeled as a linear function of different ranks; based on the historical data, forward propagation estimation of different rank configurations is performed to determine the performance gain; and based on the client weight factors, the performance gain, and the resource cost, the utility function corresponding to each of the plurality of candidate ranks for each client is determined.

[0061] In some embodiments of this application, the determining module 202 is further configured to determine the memory capacity corresponding to each of the multiple candidate ranks of each client and the number of floating-point operations corresponding to each of the multiple candidate ranks of each client; when the memory capacity is less than the memory resources and the number of floating-point operations is less than the computing resources, perform maximum rank screening based on the utility functions corresponding to each of the multiple candidate ranks of each client to determine the maximum rank of each client; perform low-rank matrix configuration based on the maximum rank of each client to determine the local matrix corresponding to each of the multiple clients; and determine the quantization level corresponding to the local matrix based on the local matrix corresponding to each of the multiple clients.

[0062] In some embodiments of this application, the determining module 202 is further configured to: determine the large language model weights corresponding to each of the plurality of clients; optimize the local matrix based on the large language model weights corresponding to each of the plurality of clients to determine the local model corresponding to each of the plurality of clients; and quantize the local model based on the quantization level corresponding to each of the plurality of clients to determine the large language model.

[0063] In some embodiments of this application, the determining module 202 is further configured to: aggregate and update low-rank adaptations of different ranks using singular value decomposition based on the model parameters corresponding to each of the plurality of clients, to obtain the updated model parameters corresponding to each of the plurality of clients; perform model training based on the updated model parameters corresponding to each of the plurality of clients until the number of training iterations reaches a preset number, to obtain the optimal model parameters corresponding to each of the plurality of clients; and perform global parameter aggregation on the optimal model parameters corresponding to each of the plurality of clients to determine the global model parameters.

[0064] Based on the above embodiments, this application also provides a large language model quantization federated fine-tuning method for heterogeneous tasks, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a large language model quantization federated fine-tuning device for heterogeneous tasks provided in an embodiment of this application. The device 3 includes a processor 301 and a memory 302. The memory 302 stores a computer program; the processor 301 retrieves and runs the computer program from the memory to execute the large language model quantization federated fine-tuning method for heterogeneous tasks as described in the above embodiment.

[0065] In the embodiments of this application, the processor 301 described above can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above processor function can also be other types, and the embodiments of this application do not specifically limit it.

[0066] This application provides a computer-readable storage medium storing a computer program for implementing, when executed by a processor, a method for quantizing federated fine-tuning of large language models for heterogeneous tasks as described in any of the above embodiments.

[0067] For example, the program instructions corresponding to the large language model quantization federated fine-tuning method for heterogeneous tasks in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to the large language model quantization federated fine-tuning method for heterogeneous tasks in the storage media are read or executed by an electronic device, the large language model quantization federated fine-tuning method for heterogeneous tasks as described in any of the above embodiments can be implemented.

[0068] Furthermore, in the embodiments of this application, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0069] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the embodiments in this application are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, these will not be repeated here.

[0071] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0072] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0073] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0074] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0075] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0076] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0077] The above description is merely an implementation method of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for quantized federated fine-tuning of large language models for heterogeneous tasks, characterized in that, The method includes: Obtain the available resources and training datasets for each of the multiple clients; Based on the available resources corresponding to each of the multiple clients, determine the local matrix and quantization level corresponding to each of the multiple clients; Based on the quantization level and local matrix corresponding to each of the multiple clients, the large language model corresponding to each of the multiple clients is determined; and the large language model of each of the multiple clients is trained using the training dataset corresponding to each of the multiple clients to obtain the model parameters corresponding to each of the multiple clients. Based on the model parameters corresponding to each of the multiple clients, global parameter aggregation is performed to determine global model parameters; so as to enable quantitative federated fine-tuning based on the global model parameters.

2. The method according to claim 1, characterized in that, The step of determining the local matrix and quantization level corresponding to each of the multiple clients based on their respective available resources includes: Based on the available resources corresponding to each of the multiple clients, feature analysis is performed to determine multiple candidate ranks corresponding to each of the multiple clients; For each of the plurality of clients, calculate the utility function corresponding to each of the plurality of candidate ranks for each client; Based on the utility functions corresponding to the multiple candidate ranks of each client, maximum rank screening is performed to determine the local matrix and quantization level corresponding to each of the multiple clients.

3. The method according to claim 2, characterized in that, The available resources include memory resources and computing resources; The step of performing feature analysis based on the available resources corresponding to each of the multiple clients to determine multiple candidate ranks corresponding to each of the multiple clients includes: Based on the memory resources corresponding to each of the multiple clients, feature analysis is performed to determine the memory characteristics corresponding to each of the multiple clients; wherein, the memory characteristics represent the free memory capacity of the client; Based on the computing resources corresponding to each of the multiple clients, feature analysis is performed to determine the computing characteristics corresponding to each of the multiple clients; wherein, the computing characteristics characterize the computing speed of the client; Based on the memory characteristics and computational characteristics of each of the multiple clients, resource analysis is performed to determine multiple candidate ranks for each of the multiple clients.

4. The method according to claim 2, characterized in that, The step of calculating the utility function corresponding to each of the multiple candidate ranks for each of the multiple clients includes: For each of the plurality of clients, historical data, client weighting factors, and resource costs are obtained; wherein, the resource cost representation is modeled as a linear function of different ranks; Based on the historical data, forward propagation estimation is performed for different rank configurations to determine the performance gain; Based on the client weight factor, the performance gain, and the resource cost, the utility function corresponding to each of the multiple candidate ranks for each client is determined.

5. The method according to claim 2, characterized in that, The process of performing maximum rank filtering based on the utility functions corresponding to the multiple candidate ranks of each client, and determining the local matrix and quantization level corresponding to each of the multiple clients, includes: Determine the memory capacity corresponding to each of the multiple candidate ranks of each client and the number of floating-point operations corresponding to each of the multiple candidate ranks of each client; When the memory capacity is less than the memory resources and the number of floating-point operations is less than the computing resources, the maximum rank of each client is determined by filtering based on the utility functions corresponding to the multiple candidate ranks of each client. Based on the maximum rank of each client, a low-rank matrix is ​​configured to determine the local matrix corresponding to each of the multiple clients. Based on the local matrices corresponding to each of the multiple clients, the quantization level corresponding to the local matrix is ​​determined.

6. The method according to claim 1, characterized in that, The step of determining the large language model corresponding to each of the multiple clients based on the quantization level and the local matrix includes: Determine the large language model weights corresponding to each of the multiple clients; and optimize the local matrix based on the large language model weights corresponding to each of the multiple clients to determine the local model corresponding to each of the multiple clients. Based on the quantization levels corresponding to each of the multiple clients, the local model is quantized to determine the large language model.

7. The method according to claim 1, characterized in that, The step of performing global parameter aggregation based on the model parameters corresponding to each of the multiple clients to determine the global model parameters includes: Based on the model parameters corresponding to each of the multiple clients, the low-rank adaptations of different ranks are aggregated and updated using the singular value decomposition method to obtain the updated model parameters corresponding to each of the multiple clients. Based on the updated model parameters corresponding to each of the multiple clients, model training is performed until the number of training iterations reaches a preset number, and the optimal model parameters corresponding to each of the multiple clients are obtained. The global model parameters are determined by performing global aggregation on the optimal model parameters corresponding to each of the multiple clients.

8. A large language model quantization federated fine-tuning system for heterogeneous tasks, characterized in that, The large language model quantization federated fine-tuning system for heterogeneous tasks includes: an acquisition module, a determination module, and a training module, wherein... The acquisition module is used to acquire the available resources and training datasets corresponding to each of the multiple clients; The determining module is used to determine the local matrix and quantization level corresponding to each of the multiple clients based on the available resources corresponding to each of the multiple clients. The training module is used to determine the large language model corresponding to each of the multiple clients based on the quantization level and the local matrix corresponding to each of the multiple clients; and to train the large language model of each of the multiple clients respectively using the training dataset corresponding to each of the multiple clients to obtain the model parameters corresponding to each of the multiple clients. The determining module is further configured to perform global parameter aggregation based on the model parameters corresponding to each of the multiple clients, and determine global model parameters, so as to enable quantitative federated fine-tuning based on the global model parameters.

9. A large language model quantization federated fine-tuning device for heterogeneous tasks, characterized in that, include: Processor and memory, of which, The memory is used to store computer programs; The processor is configured to call and run the computer program from the memory to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the method of any one of claims 1 to 7.

Images