Efficient federated learning large model training method based on dynamic layer selection
By dynamically selecting high-important layers in federated learning for training and transferring part of the computing pressure to the server, the problems of insufficient computing resources on the client and large bandwidth consumption of gradient upload are solved, and efficient federated learning training is achieved.
Patent Information
- Application Number
- CN202410907389.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-08
AI Technical Summary
In a federated learning environment, insufficient client computing resources and large gradient upload bandwidth consume, resulting in excessive computing and communication pressure, which makes it difficult for the existing technology to effectively solve this problem.
Using an efficient federated learning method based on dynamic layer selection, the server quantifies model parameters and freezes the linear layer, selects the layer with high importance for training, and transfers part of the calculation pressure to the server side. The client only updates the parameters of the specified layer to reduce the Computational and communication burden.
It significantly reduces the computing and communication pressure of the client, improves training efficiency, and does not require additional modifications to the model, and has good scalability.
Smart Images

Figure CN119026707A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of efficient training of federated learning, and in particular to an efficient federated learning large model training method based on dynamic layer selection. Background Art
[0002] In recent years, large language models have gained attention for their superior capabilities in natural language processing tasks. Powered by deep learning algorithms, these models have demonstrated superior performance in a variety of applications, including machine translation, text generation, and question-answering systems. The effectiveness of large models is largely due to their ability to leverage large amounts of high-quality data, enabling them to capture complex language patterns and semantics. As computing power has increased and the amount of data has grown, large models have continued to grow in size, enabling them to learn more nuanced and richer language representations. This increase in size has further improved the models’ generalization capabilities and performance on multiple tasks, driving advances in the field of natural language processing.
[0003] However, in certain fields, obtaining high-quality data to train large models poses a significant challenge due to the critical issue of data privacy. For example, suppose multiple hospitals want to jointly train a large diagnostic model. Patients' medical records are crucial to training such a model, but these records are highly sensitive and private. Due to privacy and regulatory constraints, traditional centralized training methods, i.e. aggregating data on a single server, are not feasible. In addition, in industries such as finance, law, and government, data privacy and security also pose a significant obstacle to training large language models. Data in these fields often contain sensitive information, such as personally identifiable information, trade secrets, or national security information, whose disclosure or misuse may have serious consequences. Therefore, it becomes critical to develop techniques that can effectively train models while protecting data privacy.
[0004] Federated learning is a distributed machine learning technique in which the server only receives the gradients uploaded by users to update the global model. Since the parameter server does not have direct access to user data, it is generally believed that the privacy of user data is protected. Therefore, federated learning can be used in scenarios such as financial services, medical prediction, speech recognition, user analysis, keyboard prediction, etc. where privacy needs to be protected. Recent research has explored combining large models with federated learning, which can train powerful models while respecting data privacy. In addition, federated learning also provides a way to reduce the demand for data storage and processing on central servers, thereby reducing dependence on computing resources. In federated learning, the distributed nature of data also helps to enhance the generalization ability of the model because it can learn from multiple data sources, thereby improving the adaptability and robustness of the model in different environments.
[0005] However, training large models in a federated setting encounters several challenges. First, the large number of parameters that large models possess requires a large amount of computational resources, which may exceed the capabilities of client devices. Second, transmitting gradients from the client to the server places a significant burden on the client's upload bandwidth. In real-world scenarios, this imbalance is further exacerbated because servers typically have powerful enterprise-grade graphics cards, while clients are equipped with less powerful consumer-grade graphics cards. In federated learning, the computational load of gradient calculations falls primarily on the client, while the server's role is limited to aggregating gradients and updating model parameters, resulting in an uneven distribution of computational resources.
[0006] In order to solve the problem of high computing pressure on the client, a feasible solution is to use the parameter-efficient fine-tuning solution. This solution uses the LoRA strategy to connect a new low-parameter module in parallel next to the weights of the original model to simulate changes to the original parameters. Although this solution can greatly reduce the amount of calculation required for training, it is still a computationally intensive process for the client, and this solution cannot solve the problem of uneven computing load between the server and the client. In order to solve the problem of high communication pressure on the client upload, a feasible solution is to compress the gradient data. However, considering the large number of parameters in the model, the size of the compressed data is still very large. It can be seen that although some technical solutions have touched on related issues to a certain extent, the effect is still not significant, and it is difficult to solve the above problems at the same time. Summary of the invention
[0007] The present invention provides an efficient federated learning large model training method based on dynamic layer selection, which distributes part of the computing pressure to the server to significantly reduce the computing pressure and communication pressure of the client.
[0008] The technical solution adopted by the present invention is:
[0009] An efficient federated learning large model training method based on dynamic layer selection performs the following steps in a federated learning system including a server and several clients:
[0010] Step 1: The server determines a model to be trained, which is a large language model;
[0011] The server quantizes the model parameters of the model to be trained (the specific quantization bit can be set based on the actual application requirements, and 4-bit quantization is recommended to save video memory to the greatest extent) to significantly reduce the data size of the model and optimize the storage and processing efficiency of the model; and freezes all linear layers of the model to be trained (that is, the fully connected layers of the model) to ensure that these layers will not be updated in the subsequent training process;
[0012] The server fine-tunes the model parameters of the linear layer of the training model based on the LoRA strategy, that is, in the subsequent federated model training, the network model parameters fine-tuned according to the LoRA strategy are updated;
[0013] The server then broadcasts the fine-tuned model to be trained to all clients participating in the training to ensure that each client can start training under the same initial conditions;
[0014] Step 2: The server performs dynamic layer selection based on the preliminary training results of the current model to be trained, and sends the layer number of the network layer of the selected model to be trained to all clients participating in the training;
[0015] The server performs dynamic layer selection based on the preliminary training results of the current model to be trained, specifically including:
[0016] The server performs preliminary training on the model to be trained based on local training data, and obtains the gradient value of each network layer of the model to be trained;
[0017] For the linear layers in the model to be trained, the importance score I of each layer is calculated based on the gradient value. l , the importance score I l It is used to characterize the importance of each layer in the initial training of the model; the amount of local data involved in the initial training does not exceed the specified value;
[0018] The server selects the first α highest importance scores I l The corresponding linear layer and the last β lowest importance scores I l The corresponding linear layer is used as the currently selected dynamic layer, where the values of α and β are preset values, and α and β are less than or equal to the total number of network layers of the model to be trained;
[0019] Step 3: After receiving the layer number specified by the server, the client performs local training on the current model to be trained based on the client's local training data set, calculates the gradient of the specified network layer as the client's update gradient and sends it back to the server;
[0020] Step 4: The server aggregates the updated gradients sent by the client to obtain the aggregated global gradients of each designated network layer, and then obtains the global model of the current model to be trained in the current round of federated learning based on the gradient values of each non-designated network layer of the current model to be trained obtained by the server in the preliminary training, and sends it to all clients participating in the training to obtain the model to be trained in the next round of federated learning;
[0021] Repeat steps 2 to 4 for multiple rounds of federated learning, and stop when the preset training convergence conditions are met (such as the number of federated learning rounds reaches the preset upper limit or the loss value of the model converges), thereby obtaining a target model that achieves the expected training objectives and performance standards.
[0022] Furthermore, in step 2, the importance score of each layer is I l The specific calculations include:
[0023] Before preliminary training, the server derives all LoRA parameters of the model to be trained to obtain the gradients ΔB and ΔA of the two LoRA parameters A and B of the LoRA strategy; the output of each linear layer of the model to be trained represented by the LoRA parameters A and B is: h = W0x + BAx, x represents the input data of the current linear layer, and W0 is the LoRA weight of the current linear layer; after the preliminary training is completed, the importance score I is calculated according to the update degree of each linear layer l ;
[0024] in,
[0025]
[0026]
[0027] In the above formula, r i represents the update degree of the i-th network layer using the LoRA strategy, that is, the update degree of the i-th linear layer, P l represents the set of all linear layers that use the LoRA strategy in the model to be trained, |P l | represents the set P l The number of linear layers included, τ is a preset constant used to avoid division by zero.
[0028] Furthermore, in step 4, when obtaining the global model of the current round of federated learning, the gradient values of each non-specified network layer of the current model to be trained obtained by the server in the preliminary training are multiplied by a scaling factor γ, whose value range is (0,1) and is recommended to be set to 0.1. The global gradients of each specified network layer after aggregation are then combined to obtain the global model of the current round of federated learning, so as to avoid the serious overfitting problem of the model caused by directly using the gradient values obtained by the local preliminary training of the server.
[0029] The technical solution provided by the present invention brings at least the following beneficial effects:
[0030] In the present invention, the server selects some of the most important layers and hands them over to the client for training, that is, part of the computational load is allocated to the server, and the client only needs to update the parameters of a very small part of the layers, thereby greatly reducing the computational pressure and upload pressure on the client, and significantly reducing the computational pressure and communication pressure on the client; in the present invention, the server uses a small part of auxiliary data to pre-calculate the gradients of all parameters of the model, and the client only trains some of the most important layers according to the calculation results, thereby greatly reducing the computational requirements for training and the parameter amount of uploaded data. At the same time, the present invention does not require additional changes to the model, and thus has strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0032] Figure 1 It is a schematic diagram of the overall process of an efficient federated learning large model training method based on dynamic layer selection provided by an example of the present invention;
[0033] Figure 2 is a schematic diagram of a method for calculating the importance score of each fully connected layer used in an example of the present invention;
[0034] Figure 3 It is a flowchart of an efficient federated learning large model training method based on dynamic layer selection provided by an example of the present invention. DETAILED DESCRIPTION
[0035] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described in detail and completely in conjunction with the drawings in the implementation of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present invention.
[0036] The embodiment of the present invention proposes an efficient federated learning large model training method based on dynamic layer selection, aiming to build a large model federated learning training framework with low client computing pressure and communication pressure, thereby realizing the training of large language models without infringing user privacy.
[0037] As a possible implementation, an efficient federated learning large model training method based on dynamic layer selection provided in an embodiment of the present invention includes the following steps:
[0038] Step 1: Prepare the model to be trained:
[0039] In the initial stage of federated learning, the server determines a large language benchmark model, which will be used as the starting model of the federated learning process and will be used in all subsequent training stages; in order to optimize the storage and processing efficiency of the model, the server quantizes the model to 4 bits, which significantly reduces the data size of the model. In addition, all fully connected layers will be frozen to ensure that these layers will not be updated during subsequent training; the server also integrates the LoRA module into the model, which is designed to reduce the number of parameters in the model training process, which means that only the LoRA module will be updated in subsequent training; finally, the model after these preprocessing steps will be broadcast by the server to all clients participating in the training, ensuring that each client can start training under the same initial conditions.
[0040] Step 2: The server performs dynamic layer selection:
[0041] The server uses a small amount of local data (local auxiliary data) to perform preliminary training on the model. The main purpose of this stage is to evaluate the performance of each layer in the model, especially to record and analyze the gradient value of each layer; based on the gradient value of each layer, the server calculates the importance score of each layer I l , these scores reflect the importance of each layer in model training and are ranked accordingly;
[0042] After sorting, the server will select the first α (experience value) layer with the highest importance score and the last β (experience value) layer with the lowest score, considering these layers to be the most in need of training or the most likely to need further optimization. The layer numbers of these layers are then sent to each client to guide them to conduct targeted training.
[0043] Step 3: Train the model locally on the client:
[0044] After receiving the layer number specified by the server, the client will use its own local data set to train these specified layers based on this information. That is, the model to be trained is forward propagated on the local training data set, but only the gradient of the specified layer is calculated when calculating the gradient in the back propagation. The client can ignore the unspecified layers during the back propagation and only calculate the gradient of the specified layer; after the training is completed, the client calculates the difference between the weights before and after training. This difference represents the update amount of the model and is a direct product of local training; then, the client sends these updates back to the server as key data in the global model update process, that is, the client update gradient.
[0045] Step 4, the server aggregates the gradients and updates the global model:
[0046] After receiving the gradients submitted by all clients, the server aggregates these data to form a global gradient. This step is the core model synchronization process in federated learning, ensuring that the training results of all clients are integrated;
[0047] Using the aggregated global gradients, the server can generate an updated global model by updating the model parameters using the Adam optimizer.
[0048] After the update is completed, the server will start the next round of federated learning and repeat the above steps until the entire model training process converges and finally achieves the expected training objectives and performance standards.
[0049] like Figure 1 As shown, the core idea of the method of the embodiment of the present invention is that before each round of federated learning begins, the server uses a small amount of auxiliary data to pre-calculate the gradients of all model parameters. These pre-calculated gradients are used to evaluate the relative importance of each layer of the model in the current learning context. Based on this evaluation, when the client uses its local data for gradient calculation, it only updates the partial layers specified by the server, rather than all parameters of the entire model. The server then combines the received client gradients with its own pre-calculated gradients to update the global model.
[0050] The present invention divides the large language model into several layers according to the Transformer decoder layer. The key issue is how to measure the importance of each layer according to the gradient, and select the layer to be trained accordingly. In the fine-tuning training of the large model, before each round of federated learning begins, the server first derives all LoRA parameters, that is, for all linear layers h=W0x+BAx, the gradients ΔB and ΔA of the LoRA parameters B and A can be obtained, where W0 is the layer weight parameter under the LoRA strategy (referred to as LoRA weight), and x represents the input data of the current layer. Since LoRA simulates the change in the original linear layer, the change in the original parameter (weight parameter) in the current training round ΔW t It can be estimated by:
[0051] ΔW t =B·ΔB+A·ΔA
[0052] Since the parameter values of different layers are not consistent, it is not fair to directly compare the parameter changes. In the current training round, the original parameters W of the linear layer are t It can be calculated by W0, B and A. Therefore, the embodiment of the present invention uses the ratio r of the parameter change to the original parameter value to measure the update degree of a linear layer:
[0053]
[0054] The overall principle is as follows Figure 2 As shown, τ is a minimum value (i.e., a preset constant) to avoid the problem of division by zero. At the same time, since the original parameters of the model are quantized using 4 bits, they need to be restored to float values before calculation for further calculation. For the Transformer decoder layer of each model (i.e., each layer of the model to be trained), the L1 norm of the r values of all linear layers therein can be calculated, and the average value can be counted to measure the parameter change of the entire decoder layer. In this embodiment of the present invention, it is assumed that for decoder layers with larger update amplitudes (r), their contributions in the current training stage are also greater, so it can be considered that these layers are more important in the current training stage. Therefore, the importance of the lth layer I l It can be calculated using the following formula:
[0055]
[0056] Among them, P l represents the set of all parameters using LoRA in the lth layer of the large model, and |P l | represents the number of parameters in the set. In this way, the importance of each Transformer decoder layer can be calculated on the server side. Subsequently, the client will only calculate the gradients for the most important layers and upload them to the server. It is worth emphasizing that the small amount of auxiliary data sets used on the server side is only used to evaluate the importance of each layer, and the actual training process still relies on the client's local data. Through this mechanism, the number of parameters that the client needs to train will be significantly reduced, and the amount of gradient data uploaded to the server will also be greatly reduced.
[0057] However, since this method tends to select layers with relatively large gradient values, a problem may arise: some layers may fall into local optimal solutions, resulting in abnormally low gradient values and inability to jump out of the local optimal point for update. To avoid this, when selecting layers, the embodiment of the present invention selects the α layer with the largest importance I and the β layer with the smallest importance I as the layers for which the client needs to calculate gradients. This strategy can effectively ensure that each layer is fully trained.
[0058] At the same time, during training, it is hoped that the gradient values calculated by the server using the auxiliary data will be fully utilized instead of being wasted. However, since the size of the auxiliary data set is relatively small and its data distribution may be inconsistent with the real user data, directly using these gradient values may cause serious overfitting problems in the model, thereby affecting the performance of the final model. To solve this problem, an embodiment of the present invention only uses the gradients calculated by the server to update the layers that have not been trained by the client in the current round of federated learning, that is, those relatively less important layers. At the same time, an embodiment of the present invention multiplies the gradients calculated by the server by a scaling factor γ to minimize its impact on the model. In this way, the global model of an embodiment of the present invention can still be updated, and more update directions and possibilities can be explored. Finally, the server will use all the collected gradients to update the global model using the Adam optimizer. The specific process is as follows: Figure 3 shown.
[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
[0060] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.
Claims
1. An efficient federated learning large model training method based on dynamic layer selection, characterized in that: The following steps are performed in a federated learning system consisting of a server and several clients: Step 1: The server determines a model to be trained, which is a large language model; The server quantizes the model parameters of the model to be trained and freezes all linear layers of the model to be trained; The server fine-tunes the model parameters of all linear layers of the training model based on the LoRA strategy for efficient parameter fine-tuning; The server then broadcasts the fine-tuned model to be trained to all clients participating in the training; Step 2: The server performs dynamic layer selection based on the preliminary training results of the current model to be trained, and sends the layer number of the network layer of the selected model to be trained to all clients participating in the training; The server performs dynamic layer selection based on the preliminary training results of the current model to be trained, specifically including: The server performs preliminary training on the model to be trained based on local training data, and obtains the gradient value of each network layer of the model to be trained; For each linear layer of the model to be trained, the importance score I of each layer is calculated based on the gradient value l ; The amount of local data involved in the preliminary training does not exceed the specified value; The server selects the first α highest importance scores I l The corresponding linear layer and the last β lowest importance scores I l The corresponding linear layer is used as the currently selected dynamic layer, where the values of α and β are preset values, and α and β are less than or equal to the total number of network layers of the model to be trained; Step 3: After receiving the layer number specified by the server, the client performs local training on the current model to be trained based on the client's local training data set, calculates the gradient of the specified network layer as the client's update gradient and sends it back to the server; Step 4: The server aggregates the updated gradients sent by the client to obtain the aggregated global gradients of each designated network layer, and then obtains the global model of the current model to be trained in the current round of federated learning based on the gradient values of each non-designated network layer of the current model to be trained obtained by the server in the preliminary training, and sends it to all clients participating in the training to obtain the model to be trained in the next round of federated learning; Repeat steps 2 to 4 for multiple rounds of federated learning and stop when the preset training convergence conditions are met.
2. The method according to claim 1, characterized in that In step 1, when the server quantizes the model parameters of the model to be trained, the quantization bit used is 4 bits.
3. The method according to claim 1, characterized in that In step 2, the importance score of each layer is I l The specific calculations include: Before preliminary training, the server derives all LoRA parameters of the model to be trained to obtain the gradients ΔB and ΔA of the two LoRA parameters A and B of the LoRA strategy; the output of each linear layer of the model to be trained based on parameters A and B is: h W0x+BA x, x represents the input data of the current linear layer, and W0 is the LoRA weight of the current linear layer; after the preliminary training is completed, the importance score I is calculated according to the update degree of each linear layer l ; in, Among them, r i represents the update degree of the i-th linear layer using the LoRA strategy, P l Represents the set of all linear network layers that use the LoRA strategy in the model to be trained, |P l | represents the set P l The number of linear layers included, τ is a preset constant used to avoid division by zero.
4. The method according to claim 1, characterized in that In step 4, when obtaining the global model of the current round of federated learning, the gradient values of each non-specified network layer of the current model to be trained obtained by the server in the preliminary training are multiplied by a scaling factor γ, where the value range of the scaling factor γ is (0,1); and then the global gradients of each specified network layer after aggregation are combined to obtain the global model of the current round of federated learning.
5. The method according to claim 4, characterized in that: The scaling factor γ is set to 0.1.
Citation Information
Patent Citations
Federal learning method and system based on customer selection and weight distribution, and medium
CN116776948A
Client selection and adaptive model aggregation method and system based on reinforcement learning in federated learning
CN117459570A
Image classification device and algorithm based on federal transfer learning
CN117593582A
Lightweight user privacy protection system and method based on federal learning
CN118013585A
Federated Learning with Adaptive Optimization
US20210073639A1
