Gradient projection large model federal fine tuning method and system with efficient communication

Abstract

The invention discloses a communication-efficient gradient projection large model federal fine tuning method and system, and relates to the technical field of distributed computer federal learning. The method comprises the following steps: constructing a communication-efficient large-scale basic model federal fine tuning framework based on gradient projection; the client calculates the gradient of each layer through first-order back propagation, and further calculates the sensitivity of each layer; generating a projection matrix based on the random seed and performing dimension reduction to obtain a vector after projection; adding Gaussian noise to the projected vector to obtain a vector added with noise; forming a data packet based on the sensitivity, the random seed and the vector added with the noise, and uploading the data packet to a server; and the server updates global model parameters through a global momentum mechanism based on the data of the data packet until the global model reaches a preset precision convergence condition on the verification set, and fine tuning is completed. According to the invention, the communication efficiency of full-parameter fine tuning of a large-scale basic model in a federated environment can be improved.

Description

A communication-efficient method and system for federated fine-tuning of large gradient projection models Technical Field

[0001] This invention relates to the field of distributed computer federated learning technology, and in particular to a method and system for federated fine-tuning of large gradient projection models with high communication efficiency. Background Technology

[0002] In recent years, large-scale pre-trained models based on the Transformer architecture, such as large language models, have demonstrated remarkable versatility in natural language processing and multimodal understanding. To adapt these general-purpose models to specific downstream tasks (such as medical consultation, legal analysis, or personal assistants), "pre-training + downstream fine-tuning" has become the standard paradigm in industry. Typically, the private data for downstream tasks is distributed on edge devices, and directly aggregating this data to the cloud poses serious privacy risks and compliance issues. Federated learning, as a distributed machine learning framework, allows clients to collaboratively train a global model by exchanging model gradients or parameter updates without sharing the original data, thus breaking down "data silos" while protecting data privacy.

[0003] Fine-tuning the base model in a large-scale federated learning environment faces severe communication challenges. A typical 7B parameter model occupies approximately 28GB of space for its full gradients in 32-bit floating-point format. In current mobile communication networks, frequent uploads of gigabyte-level updates from hundreds or thousands of clients can cause severe network congestion and extremely high packet loss rates, leading to an exponential increase in training latency. This contradiction between communication bandwidth limitations and massive parameter updates constitutes the "communication wall" that hinders the implementation of federated learning in the field of large-scale base models.

[0004] To address the need for fine-tuning large-scale pre-trained models in a federated environment, there is an urgent need for a novel fine-tuning method that can sense the sensitivity of each layer of the model and dynamically allocate communication resources, while ensuring accuracy and achieving extremely high communication efficiency. This would enable efficient full-parameter performance tuning of large-scale pre-trained models in resource-constrained edge networks.

[0005] To alleviate communication pressure, existing technologies have attempted the following approaches, but all have limitations: (1) Efficient parameter fine-tuning: such as using algorithms like LoRA and Adapter to reduce transmission volume by updating only a very small number of external parameters. However, in complex scenarios with extremely large-scale nodes or requiring full parameter distribution accuracy, efficient parameter fine-tuning may still face performance bottlenecks.

[0006] (2) Gradient compression and quantization: The volume can be reduced by sparsifying or quantizing the gradient with low bit, but this often introduces a non-negligible approximation error, which leads to unstable convergence of the model in complex downstream tasks.

[0007] (3) Zero-order optimization: such as the MeZO algorithm, which estimates the gradient only by the loss difference of forward propagation. Although it avoids backpropagation and saves memory, its search efficiency in ultra-high-dimensional parameter space is extremely low, which leads to a surge in the number of communication rounds required for a single convergence, and the overall communication efficiency is not substantially improved.

[0008] Current technologies, especially in multi-user and highly interference-prone federated environments, lack sophisticated and adaptive allocation mechanisms for communication resources (such as TTI length and projection dimension). Furthermore, in pursuing communication compression, existing solutions often neglect the systematic integration of privacy protection (such as differential privacy) for efficient deployment in low-dimensional spaces. Summary of the Invention

[0009] To address the technical problem of low communication efficiency caused by the massive number of model parameters when performing full-parameter fine-tuning of large-scale foundational models in a federated environment, this invention provides a highly efficient gradient projection federated fine-tuning method and system for large models. The technical solution is as follows:

[0010] On the one hand, a communication-efficient gradient projection large model federated fine-tuning method is provided. This method is implemented by a communication-efficient gradient projection large model federated fine-tuning device. The method includes: S1, the server initializes the basic model parameters and global momentum; presets the total projection dimension budget and privacy strength; S2, the client performs local computation. For multiple selected clients, each client synchronizes the current global model parameters; each client calculates the gradient of each layer based on its local private dataset through first-order backpropagation; the gradients of each layer are input into the gradient sensitivity quantization module to calculate the sensitivity of each layer; S3, the sensitivity of each layer is input into the adaptive dimension random projection module to calculate the projection dimension of each layer; the client generates mutually orthogonal random projection matrices based on the random seed and the projection dimensions of each layer, and performs dimensionality reduction mapping on the random projection matrices to output the projected vector; S4, the projected vector is input into the differential privacy and seed compression module. In the compression module, Gaussian noise is added to the projected vector to obtain the projected vector with added Gaussian noise; the random seed, the sensitivity of each layer, and the projected vector with added Gaussian noise are combined into a triplet data packet and compressed; the compressed data packet is obtained; the compressed data packet is sent to the server; S5, the server receives the compressed data packet uploaded by the server and performs a global update; the received random seed is input into the global gradient reconstruction and momentum acceleration aggregation module to recover the random projection matrix and reconstruct the global gradient estimate of each layer; based on the momentum information of the previous round and the reconstructed global gradient estimate, the momentum information of the current round is calculated; based on the momentum information of the current round, the global model parameters are updated using the gradient descent method; S6, steps S1-S5 are repeated iteratively until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, at which point the iteration stops and the fine-tuned model is output.

[0011] On the other hand, a highly efficient gradient projection large model federated fine-tuning system is provided. This system is applied to a highly efficient gradient projection large model federated fine-tuning method. The system includes: a server for initializing basic model parameters and global variables; presetting the total projection dimension budget and privacy strength; the server receiving compressed data packets uploaded by the server and performing global updates; clients for local computation; for multiple selected clients, each client synchronizing the current global model parameters; each client calculating the gradients of each layer based on its local private dataset through first-order backpropagation; a gradient sensitivity quantization module for calculating the sensitivity of each layer; an adaptive dimension random projection module for calculating the projection dimension of each layer based on the sensitivity of each layer; and the client generating mutually orthogonal random projection matrices based on a random seed and the projection dimensions of each layer, and performing dimensionality reduction on the random projection matrices. The system performs a dimensional mapping to output the projected vector. A differential privacy and seed compression module adds Gaussian noise to the projected vector to obtain a Gaussian-noise-added projection vector. It then compresses a triplet data packet containing the random seed, the sensitivity of each layer, and the Gaussian-noise-added projection vector. The compressed data packet is then sent to the server. A global gradient reconstruction and momentum acceleration aggregation module recovers the random projection matrix from the received random seed input and reconstructs the global gradient estimate for each layer. Based on the momentum information from the previous round and the reconstructed global gradient estimate, it calculates the momentum information for the current round. Based on the momentum information for the current round, it updates the global model parameters using gradient descent. The iteration continues until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, at which point the iteration stops and the fine-tuned model is output.

[0012] On the other hand, a communication-efficient gradient projection large model federated fine-tuning device is provided, the communication-efficient gradient projection large model federated fine-tuning device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for communication-efficient gradient projection large model federated fine-tuning is implemented.

[0013] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of the efficient gradient projection large model federated fine-tuning method.

[0014] The beneficial effects of the technical solution provided by the embodiments of the present invention include at least the following: The embodiments of the present invention propose a communication-efficient large-scale foundation model federated fine-tuning framework based on gradient projection. This scheme avoids the problem of excessively large variance in gradient estimation in high-dimensional space by traditional zero-order optimization methods. Based on first-order gradient information, through the collaborative cooperation between the client and server based on random seeds, communication overhead is greatly reduced while ensuring the accuracy of model fine-tuning.

[0015] This invention utilizes an adaptive dimensional random projection module to map the full gradients of billions of parameters to an extremely low-dimensional coordinate space. While maintaining fine-tuning accuracy, this invention reduces uplink communication overhead to the kilobyte level compared to traditional first-order federated learning. Simultaneously, through a sensitivity-aware mechanism, the system can identify parameter layers with varying contributions to the task and prioritize the allocation of limited communication bandwidth to critical "active layers," effectively solving the problem of non-essential parameter layers occupying redundant bandwidth due to fixed-dimensional projection in existing technologies.

[0016] This invention's embodiments rely entirely on first-order gradient information for local updates, avoiding the slow convergence problem caused by gradient estimation bias in ultra-high-dimensional parameter spaces in zero-order optimization methods. Compared to current mainstream zero-order federated learning schemes, this invention achieves approximately 20 times the speedup in the number of communication rounds required for convergence. Furthermore, due to the use of an unbiased reconstruction algorithm based on first-order gradients, the accuracy of its fine-tuned model significantly outperforms efficient parameter fine-tuning and various zero-order fine-tuning schemes, exhibiting performance closer to that of fully parameter-local training.

[0017] This invention utilizes a shared random seed mechanism, allowing the client to reconstruct gradients on the server side with only a tiny seed value and compressed scalar coordinates, significantly reducing the risk of exposing the original data and sensitive gradients. Building upon this, this invention introduces a differential privacy-focused noise-adding strategy for low-dimensional projective spaces. Compared to adding noise in the original high-dimensional space, this invention achieves the same level of privacy protection at a lower accuracy cost. Simultaneously, the application of block-based reconstruction technology significantly reduces the computational complexity on the server side, ensuring the system's scalability when handling ultra-large-scale base models.

[0018] The framework designed in this invention is highly versatile, and its modules support multiple mainstream optimization algorithms such as AdamW and SGD, enabling seamless integration into existing deep learning training pipelines. The system can dynamically adjust the projection dimension budget based on real-time network bandwidth and the computing power of edge devices, ensuring stable operation of the fine-tuning process in heterogeneous and dynamic distributed environments, demonstrating excellent engineering application prospects. This invention can improve the communication efficiency of full-parameter fine-tuning of large-scale foundational models in a federated environment. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 is a flowchart of a communication-efficient gradient projection large model federated fine-tuning method provided by an embodiment of the present invention; Figure 2 is a flowchart of a panoramic architecture and interaction process of a communication-efficient large-scale basic model federated fine-tuning based on gradient projection provided by an embodiment of the present invention; Figure 3 is a flowchart of a communication-efficient large-scale basic model federated fine-tuning framework provided by an embodiment of the present invention; Figure 4 is a block diagram of a communication-efficient gradient projection large model federated fine-tuning system provided by an embodiment of the present invention; Figure 5 is a structural schematic diagram of a communication-efficient gradient projection large model federated fine-tuning device provided by an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0022] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0023] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0024] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0025] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0026] This invention provides a communication-efficient gradient projection large model federated fine-tuning method. This method can be implemented using a communication-efficient gradient projection large model federated fine-tuning device, which can be a terminal or a server. As shown in Figure 1, the communication-efficient gradient projection large model federated fine-tuning method's flowchart includes the following steps:

[0027] S1. The server initializes the basic model parameters and global momentum; presets the total projection dimension budget and privacy strength.

[0028] This invention specifically addresses fine-tuning large-scale pre-trained models. It can be applied to environments with data privacy requirements, such as the medical and financial industries. Furthermore, this invention addresses the communication efficiency bottleneck caused by the massive number of model parameters.

[0029] S2. Client-side local computation: For multiple selected clients, each client synchronizes the current global model parameters; each client calculates the gradient of each layer based on its local private dataset through first-order backpropagation; the gradient of each layer is input into the gradient sensitivity quantization module to calculate the sensitivity of each layer.

[0030] Optionally, the process of calculating the sensitivity of each layer is expressed by the following formula (1): (1) Among them, This indicates that at iteration t, the th... The sensitivity of the layer; This indicates that at iteration t, the th... Layer Gradient of dimension; Indicates the first Total dimensions of layer parameters.

[0031] S3. Input the sensitivity of each layer into the adaptive dimension random projection module to calculate the projection dimension of each layer; the client generates mutually orthogonal random projection matrices based on the random seed and the projection dimension of each layer, performs dimension reduction mapping on the random projection matrices, and outputs the projected vector.

[0032] Optionally, the process of calculating the projection dimensions of each layer is expressed by the following formula (2): (2) Among them, This indicates the preset total projection dimension budget; This indicates that at iteration t, the th... The projection dimension of the layer; This indicates that at iteration t, the th... The sensitivity of the layer; Indicates the total number of floors.

[0033] The elements of the mutually orthogonal random projection matrices have a mean of 0 and a variance of . , a Gaussian distribution; where, Indicates the projection dimension.

[0034] Optionally, the process of dimensionality reduction mapping of the random projection matrix is ​​represented by the following formula (3): (3) Among them, This indicates that at iteration t, the th... The vector after layer projection; Indicates the first Layer random projection matrix vector; This indicates that at iteration t, the th... Full gradient of the layer.

[0035] In one feasible implementation, the present invention uses an audit sensitivity-aware adaptive dimension allocation mechanism to ensure that layers with higher sensitivity retain richer original gradient features.

[0036] S4. Input the projected vector into the differential privacy and seed compression module, add Gaussian noise to the projected vector to obtain the projected vector with added Gaussian noise; combine the random seed, the sensitivity of each layer and the projected vector with added Gaussian noise into a triplet data packet and compress it; obtain the compressed data packet; send the compressed data packet to the server.

[0037] In order to protect local data privacy, the projection vector is... Perform Gaussian noise processing to make it meet the requirements. - Differential privacy; among which, Indicates privacy budget, This represents the probability of violating strict differential privacy, or the probability of failure.

[0038] Optionally, the process of obtaining the projection vector after adding Gaussian noise is represented by the following formula (4): (4) Among them, This indicates that at iteration t, the th... The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... The projection vector of the layer; Indicates a Gaussian distribution; Indicated by privacy budget The determined noise intensity; Indicates the gradient clipping threshold; Represents the identity matrix.

[0039] The compressed data packet can be represented as: .

[0040] In one feasible implementation, noise is added to the projected low-dimensional coordinate vector, and the noise intensity or pruning threshold is adaptively adjusted to minimize the impact of noise on the core gradient information while satisfying privacy budget constraints.

[0041] S5. The server receives the compressed data packet uploaded by the server and performs a global update; it inputs the received random seed into the global gradient reconstruction and momentum acceleration aggregation module to recover the random projection matrix and reconstruct the global gradient estimate of each layer; based on the momentum information of the previous round and the reconstructed global gradient estimate, it calculates the momentum information of the current round; based on the momentum information of the current round, it updates the global model parameters using the gradient descent method.

[0042] Optionally, the process of reconstructing the global gradient estimate for each layer is represented by the following formula (5): (5) Among them, Indicates the number of clients; This indicates the m-th client. Layer random projection matrix; This indicates that the m-th client is at iteration t. The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... Reconstruct the global gradient estimate for each layer.

[0043] In one feasible implementation, the momentum information of the current round is calculated using the momentum information of the previous round, which can smooth out the projection error.

[0044] Optionally, the process of updating the global model parameters using gradient descent is represented by the following formula (6): (6) Among them, This represents the parameters of the global model at iteration t; Indicates the learning rate; This indicates the momentum information at iteration t; This represents the parameters of the global model at iteration t+1.

[0045] Among them, the momentum information at iteration t It can be expressed by the following formula (7): (7) Among them, Indicates the momentum coefficient; This indicates the momentum information at iteration t-1; This indicates that at iteration t, the th... Reconstruct the global gradient estimate for each layer.

[0046] S6. Repeat steps S1-S5 iteratively until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, then stop iterating and output the fine-tuned model.

[0047] The embodiments of this invention can be applied to healthcare scenarios. They can fine-tune large-scale medical models by employing a refined and adaptive allocation mechanism for communication resources to output a finely tuned model, thereby protecting patient privacy and improving the accuracy of disease diagnosis. The embodiments of this invention can also be applied to the financial industry, where they can fine-tune interbank risk control models to prevent systemic risks.

[0048] In one feasible implementation, Figure 2 shows a panoramic architecture and interaction flowchart provided by an embodiment of the present invention. The central server includes a global gradient reconstruction and momentum acceleration aggregation module. The client includes a gradient sensitivity quantization module, an adaptive dimensional random projection module, and a differential privacy and seed compression module. The computational steps involved include local private data, first-order gradient training, adaptive projection dimension calculation, and differential privacy noise addition. The communication path is a downlink for transmitting the global model, and an uplink for transmitting only lightweight random seeds, dimension configurations, and low-dimensional projection gradients, significantly reducing overhead.

[0049] In one feasible implementation, Figure 3 is a system framework flowchart provided by an embodiment of the present invention; wherein, the specific process includes: device and parameter initialization; the client executes the steps of downloading the model, gradient calculation, sensitivity evaluation, dynamic projection, privacy noise addition and data upload; the server executes the steps of gradient recovery, heterogeneous aggregation, momentum optimization and parameter update; finally, the closed loop is completed by reaching convergence through iterative looping or reaching the maximum number of communication rounds.

[0050] This invention proposes a communication-efficient large-scale foundation model federated fine-tuning framework based on gradient projection. This scheme avoids the problem of excessively large variance in gradient estimation in high-dimensional spaces caused by traditional zero-order optimization methods. Based on first-order gradient information, and through collaborative cooperation between the client and server using random seeds, it significantly reduces communication overhead while ensuring model fine-tuning accuracy.

[0051] This invention utilizes an adaptive dimensional random projection module to map the full gradients of billions of parameters to an extremely low-dimensional coordinate space. While maintaining fine-tuning accuracy, this invention reduces uplink communication overhead to the kilobyte level compared to traditional first-order federated learning. Simultaneously, through a sensitivity-aware mechanism, the system can identify parameter layers with varying contributions to the task and prioritize the allocation of limited communication bandwidth to critical "active layers," effectively solving the problem of non-essential parameter layers occupying redundant bandwidth due to fixed-dimensional projection in existing technologies.

[0052] This invention's embodiments rely entirely on first-order gradient information for local updates, avoiding the slow convergence problem caused by gradient estimation bias in ultra-high-dimensional parameter spaces in zero-order optimization methods. Compared to current mainstream zero-order federated learning schemes, this invention achieves approximately 20 times the speedup in the number of communication rounds required for convergence. Furthermore, due to the use of an unbiased reconstruction algorithm based on first-order gradients, the accuracy of its fine-tuned model significantly outperforms efficient parameter fine-tuning and various zero-order fine-tuning schemes, exhibiting performance closer to that of fully parameter-local training.

[0053] This invention utilizes a shared random seed mechanism, allowing the client to reconstruct gradients on the server side with only a tiny seed value and compressed scalar coordinates, significantly reducing the risk of exposing the original data and sensitive gradients. Building upon this, this invention introduces a differential privacy-focused noise-adding strategy for low-dimensional projective spaces. Compared to adding noise in the original high-dimensional space, this invention achieves the same level of privacy protection at a lower accuracy cost. Simultaneously, the application of block-based reconstruction technology significantly reduces the computational complexity on the server side, ensuring the system's scalability when handling ultra-large-scale base models.

[0054] The framework designed in this invention is highly versatile, and its modules support multiple mainstream optimization algorithms such as AdamW and SGD, enabling seamless integration into existing deep learning training pipelines. The system can dynamically adjust the projection dimension budget based on real-time network bandwidth and the computing power of edge devices, ensuring stable operation of the fine-tuning process in heterogeneous and dynamic distributed environments, demonstrating excellent engineering application prospects. This invention can improve the communication efficiency of full-parameter fine-tuning of large-scale foundational models in a federated environment.

[0055] Figure 4 is a block diagram of a communication-efficient gradient projection large model federated fine-tuning system provided by an embodiment of the present invention. This system is used for a communication-efficient gradient projection large model federated fine-tuning method. Referring to Figure 4, the system includes a server 410, a client 420, a gradient sensitivity quantization module 430, an adaptive dimensional random projection module 440, a differential privacy and seed compression module 450, and a global gradient reconstruction and momentum acceleration aggregation module 460. Wherein:

[0056] Server 410 initializes basic model parameters and global variables; presets the total projection dimension budget and privacy strength; the server receives compressed data packets uploaded by the server and performs global updates; Client 420 performs local computation; for multiple selected clients, each client synchronizes the current global model parameters; each client calculates the gradient of each layer based on its local private dataset through first-order backpropagation; Gradient sensitivity quantization module 430 calculates the sensitivity of each layer; Adaptive dimensionality random projection module 440 calculates the projection dimension of each layer based on the sensitivity of each layer; the client generates mutually orthogonal random projection matrices based on the random seed and the projection dimension of each layer, performs dimensionality reduction mapping on the random projection matrices, and outputs the projected vector; Differential privacy and seed compression module 450 The first module adds Gaussian noise to the projected vector to obtain the Gaussian-noise-added projected vector; it then compresses the random seed, the sensitivity of each layer, and the Gaussian-noise-added projected vector into a triplet data packet; it obtains the compressed data packet; and it sends the compressed data packet to the server. The second module, Global Gradient Reconstruction and Momentum Acceleration Aggregation 460, recovers the random projection matrix from the received random seed input and reconstructs the global gradient estimate of each layer; it calculates the momentum information of the current round based on the momentum information of the previous round and the reconstructed global gradient estimate; it updates the global model parameters using gradient descent based on the momentum information of the current round; and it iterates repeatedly until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, at which point it stops iterating and outputs the fine-tuned model.

[0057] Optionally, the process of calculating the sensitivity of each layer is represented by the following formula (1): (1) Among them, This indicates that at iteration t, the th... The sensitivity of the layer; This indicates that at iteration t, the th... Layer Gradient of dimension; Indicates the first Total dimensions of layer parameters.

[0058] Optionally, the process of calculating the projection dimensions of each layer is represented by the following formula (2): (2) Among them, This indicates the preset total projection dimension budget; This indicates that at iteration t, the th... The projection dimension of the layer; This indicates that at iteration t, the th... The sensitivity of the layer; Indicates the total number of floors.

[0059] Optionally, the process of dimensionality reduction mapping of the random projection matrix is ​​represented by the following formula (3): (3) Among them, This indicates that at iteration t, the th... The vector after layer projection; Indicates the first Layer random projection matrix vector; This represents the full gradient of the l-th layer at iteration t.

[0060] Optionally, the process of obtaining the projection vector after adding Gaussian noise is represented by the following formula (4): (4) Among them, This indicates that at iteration t, the th... The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... The projection vector of the layer; Indicates a Gaussian distribution; Indicated by privacy budget The determined noise intensity; Indicates the gradient clipping threshold; Represents the identity matrix.

[0061] Optionally, the process of reconstructing the global gradient estimate of each layer is represented by the following formula (5): (5) Among them, Indicates the number of clients; This indicates the m-th client. Layer random projection matrix; This indicates that the m-th client is at iteration t. The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... Reconstruct the global gradient estimate for each layer.

[0062] Optionally, the process of updating the global model parameters using gradient descent is represented by the following formula (6): (6) Among them, This represents the parameters of the global model at iteration t; Indicates the learning rate; This indicates the momentum information at iteration t; This represents the parameters of the global model at iteration t+1.

[0063] This invention proposes a communication-efficient large-scale foundation model federated fine-tuning framework based on gradient projection. This scheme avoids the problem of excessively large variance in gradient estimation in high-dimensional spaces caused by traditional zero-order optimization methods. Based on first-order gradient information, and through collaborative cooperation between the client and server using random seeds, it significantly reduces communication overhead while ensuring model fine-tuning accuracy.

[0064] This invention utilizes an adaptive dimensional random projection module to map the full gradients of billions of parameters to an extremely low-dimensional coordinate space. While maintaining fine-tuning accuracy, this invention reduces uplink communication overhead to the kilobyte level compared to traditional first-order federated learning. Simultaneously, through a sensitivity-aware mechanism, the system can identify parameter layers with varying contributions to the task and prioritize the allocation of limited communication bandwidth to critical "active layers," effectively solving the problem of non-essential parameter layers occupying redundant bandwidth due to fixed-dimensional projection in existing technologies.

[0065] This invention's embodiments rely entirely on first-order gradient information for local updates, avoiding the slow convergence problem caused by gradient estimation bias in ultra-high-dimensional parameter spaces in zero-order optimization methods. Compared to current mainstream zero-order federated learning schemes, this invention achieves approximately 20 times the speedup in the number of communication rounds required for convergence. Furthermore, due to the use of an unbiased reconstruction algorithm based on first-order gradients, the accuracy of its fine-tuned model significantly outperforms efficient parameter fine-tuning and various zero-order fine-tuning schemes, exhibiting performance closer to that of fully parameter-local training.

[0066] This invention utilizes a shared random seed mechanism, allowing the client to reconstruct gradients on the server side with only a tiny seed value and compressed scalar coordinates, significantly reducing the risk of exposing the original data and sensitive gradients. Building upon this, this invention introduces a differential privacy-focused noise-adding strategy for low-dimensional projective spaces. Compared to adding noise in the original high-dimensional space, this invention achieves the same level of privacy protection at a lower accuracy cost. Simultaneously, the application of block-based reconstruction technology significantly reduces the computational complexity on the server side, ensuring the system's scalability when handling ultra-large-scale base models.

[0067] The framework designed in this invention is highly versatile, and its modules support multiple mainstream optimization algorithms such as AdamW and SGD, enabling seamless integration into existing deep learning training pipelines. The system can dynamically adjust the projection dimension budget based on real-time network bandwidth and the computing power of edge devices, ensuring stable operation of the fine-tuning process in heterogeneous and dynamic distributed environments, demonstrating excellent engineering application prospects. This invention can improve the communication efficiency of full-parameter fine-tuning of large-scale foundational models in a federated environment.

[0068] Figure 5 is a schematic diagram of a communication-efficient gradient projection large model federated fine-tuning device provided in an embodiment of the present invention. As shown in Figure 5, the communication-efficient gradient projection large model federated fine-tuning device may include the communication-efficient gradient projection large model federated fine-tuning system shown in Figure 4 above. Optionally, the communication-efficient gradient projection large model federated fine-tuning device 510 may include a first processor 2001.

[0069] Optionally, the communication-efficient gradient projection large model federated fine-tuning device 510 may also include a memory 2002 and a transceiver 2003.

[0070] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0071] The following describes the various components of the efficient gradient projection large model federated fine-tuning device 510 with reference to Figure 5: The first processor 2001 is the control center of the efficient gradient projection large model federated fine-tuning device 510. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0072] Optionally, the first processor 2001 can perform various functions of the communication-efficient gradient projection large model federated fine-tuning device 510 by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0073] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG5.

[0074] In a specific implementation, as one example, the communication-efficient gradient projection large model federated fine-tuning device 510 may also include multiple processors, such as the first processor 2001 and the second processor 2004 shown in Figure 5. Each of these processors may be a single-core processor or a multi-core processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0075] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0076] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through the interface circuit of the SSS device 510 (not shown in FIG. 5). This embodiment of the invention does not specifically limit this.

[0077] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0078] Optionally, transceiver 2003 may include a receiver and a transmitter (not shown separately in Figure 5). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0079] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and is coupled to the first processor 2001 through the interface circuit (not shown in Figure 5) of the communication-efficient gradient projection large model federated fine-tuning device 510. This embodiment of the invention does not specifically limit this.

[0080] It should be noted that the structure of the communication-efficient gradient projection large model federated fine-tuning device 510 shown in Figure 5 does not constitute a limitation on the router. The actual communication-efficient gradient projection large model federated fine-tuning device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] Furthermore, the technical effects of the communication-efficient gradient projection large model federated fine-tuning device 510 can be referred to the technical effects of the communication-efficient gradient projection large model federated fine-tuning method described in the above method embodiments, and will not be repeated here.

[0082] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0083] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0084] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0085] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0086] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0087] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0088] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0089] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, systems, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0090] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A communication-efficient gradient projection large model federated fine-tuning method, characterized in that, The method is implemented by a communication-efficient large-scale foundation model federated fine-tuning framework based on gradient projection; the framework includes a server and a client; the client includes a gradient sensitivity quantization module, an adaptive dimensional random projection module, and a differential privacy and seed compression module; The server includes a global gradient reconstruction and momentum acceleration aggregation module; the method includes: S1, the server initializes the basic model parameters and global momentum; presets the total projection dimension budget and privacy strength; S2, the client performs local computation: for multiple selected clients, each client synchronizes the current global model parameters; each client calculates the gradient of each layer based on its local private dataset through first-order backpropagation; the gradients of each layer are input into the gradient sensitivity quantization module to calculate the sensitivity of each layer; S3, the sensitivity of each layer is input into the adaptive dimension random projection module to calculate the projection dimension of each layer; the client generates mutually orthogonal random projection matrices based on the random seed and the projection dimension of each layer, and performs dimension reduction mapping on the random projection matrices to output the projected vector; S4, the projected vector is input into the differential privacy and seed compression module to add high-order vectors to the projected vector. S1. Add Gaussian noise to obtain the projection vector after adding Gaussian noise; combine the random seed, the sensitivity of each layer, and the projection vector after adding Gaussian noise into a triplet data packet and compress it; obtain the compressed data packet; send the compressed data packet to the server; S5. The server receives the compressed data packet uploaded by the server and performs a global update; input the received random seed into the global gradient reconstruction and momentum acceleration aggregation module to recover the random projection matrix and reconstruct the global gradient estimate of each layer; calculate the momentum information of the current round based on the momentum information of the previous round and the reconstructed global gradient estimate; update the global model parameters using the gradient descent method based on the momentum information of the current round; S6. Repeat steps S1-S5 iteratively until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, then stop iterating and output the fine-tuned model.

2. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of calculating the sensitivity of each layer is expressed by the following formula (1): (1) Among them, This indicates that at iteration t, the th... The sensitivity of the layer; This indicates that at iteration t, the th... Layer Gradient of dimension; Indicates the first Total dimensions of layer parameters.

3. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of calculating the projection dimensions of each layer is expressed by the following formula (2): (2) Among them, This indicates the preset total projection dimension budget; This indicates that at iteration t, the th... The projection dimension of the layer; This indicates that at iteration t, the th... The sensitivity of the layer; Indicates the total number of floors.

4. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of dimensionality reduction mapping of the random projection matrix is ​​represented by the following formula (3): (3) Among them, This indicates that at iteration t, the th... The vector after layer projection; Indicates the first Layer random projection matrix vector; This indicates that at iteration t, the th... Full gradient of the layer.

5. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of obtaining the projection vector after adding Gaussian noise is represented by the following formula (4): (4) Among them, This indicates that at iteration t, the th... The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... The projection vector of the layer; Indicates a Gaussian distribution; Indicated by privacy budget The determined noise intensity; Indicates the gradient clipping threshold; Represents the identity matrix.

6. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of reconstructing the global gradient estimate of each layer is represented by the following formula (5): (5) Among them, Indicates the number of clients; This indicates the m-th client. Layer random projection matrix; This indicates that the m-th client is at iteration t. The projection vector of the layer after adding Gaussian noise; This indicates that at iteration t, the th... Reconstruct the global gradient estimate for each layer.

7. The efficient gradient projection large model federated fine-tuning method according to claim 1, characterized in that, The process of updating the global model parameters using gradient descent is represented by the following formula (6): (6) Among them, This represents the parameters of the global model at iteration t; Indicates the learning rate; This indicates the momentum information at iteration t; This represents the parameters of the global model at iteration t+1.

8. A communication-efficient gradient projection large model federated fine-tuning system, wherein the communication-efficient gradient projection large model federated fine-tuning system is used to implement the communication-efficient gradient projection large model federated fine-tuning method as described in any one of claims 1-7, characterized in that, The system includes: a server for initializing basic model parameters and global variables; presetting the total projection dimension budget and privacy strength; the server receiving compressed data packets uploaded by the server and performing global updates; a client for local computation; for synchronizing the current global model parameters for each selected client; each client calculating the gradient of each layer based on its local private dataset through first-order backpropagation; a gradient sensitivity quantization module for calculating the sensitivity of each layer; an adaptive dimensionality random projection module for calculating the projection dimension of each layer based on the sensitivity of each layer; the client generating mutually orthogonal random projection matrices based on a random seed and the projection dimensions of each layer, and performing dimensionality reduction mapping on the random projection matrices to output the projected vector; and a differential privacy and seed compression module for... Gaussian noise is added to the projected vector to obtain the Gaussian-noise-added projected vector; the random seed, the sensitivity of each layer, and the Gaussian-noise-added projected vector are combined into a triplet data packet and compressed; the compressed data packet is obtained; the compressed data packet is sent to the server; the global gradient reconstruction and momentum acceleration aggregation module is used to recover the random projection matrix from the received random seed input and reconstruct the global gradient estimate of each layer; based on the momentum information of the previous round and the reconstructed global gradient estimate, the momentum information of the current round is calculated; based on the momentum information of the current round, the global model parameters are updated using the gradient descent method; the iteration is repeated until the global model reaches the maximum number of communication rounds on the validation set or reaches the preset accuracy convergence condition, at which point the iteration stops and the fine-tuned model is output.

9. A highly efficient gradient projection large model federated fine-tuning device, characterized in that, The communication-efficient gradient projection large model federated fine-tuning device includes: a processor; and a memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images