An efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning

Abstract

This invention provides an efficiency-aware differential privacy noise injection method for fine-tuning direction-amplitude decomposition-based federated LoRA, relating to the field of data privacy protection technology. In each round of communication, the client performs a reparameterized local training phase and evaluates the local importance of LoRA updates across layers. The client prunes the LoRA parameter updates to be uploaded, introducing a hierarchical perturbation mechanism. Based on the importance level division, differentiated noise injection strategies are adopted for LoRA layers of different importance levels. Under the premise of satisfying differential privacy constraints, the redundant noise terms introduced by low-rank decomposition are eliminated from the structural level by deterministically mapping the injection method of the original noise in the weight space back to the LoRA factor space. The noise-reconstructed LoRA parameter updates are uploaded to the server for federated aggregation and broadcast back to each client as initialization parameters for the next round of local training.

Description

Technical Field

[0001] This invention relates to the field of data privacy protection technology, and in particular to an efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning. Background Technology

[0002] With the widespread application of large-scale foundational models in fields such as natural language processing and computer vision, efficient fine-tuning of large models in distributed environments with data privacy constraints has become a research hotspot. Federated learning, as a technical framework that enables multi-party collaborative training without centralized raw data, can protect data privacy to a certain extent and has been widely used in privacy-sensitive scenarios. Meanwhile, efficient parameter fine-tuning methods, especially low-rank fitting methods, effectively fine-tune large models by introducing only a small number of low-rank parameters, offering significant advantages in reducing computational overhead and communication costs. The Federated LoRA fine-tuning method, which combines LoRA with federated learning, shows promising application prospects in distributed large-scale model training.

[0003] Furthermore, existing directional-magnitude decomposition-based federated LoRA optimization architectures perform fine-grained decomposition of the LoRA low-rank matrices A and B, using the directional and magnitude components of the matrix vectors for learning globally shared knowledge and locally personalized knowledge, respectively. For example, in this type of architecture, the directional component of matrix A focuses more on global optimization, while the magnitude component of matrix B focuses more on locally personalized optimization.

[0004] However, in the direction-amplitude decomposition federated LoRA fine-tuning architecture, the LoRA update not only includes the product structure of low-rank matrices A and B, but also the reparameterized combination relationship between the direction and amplitude components. If differential privacy noise is directly injected into multiple low-rank factors or decomposition sub-components, multiplicative interactions and quadratic noise terms will be generated during the weight reconstruction process, leading to noise amplification.

[0005] Differential privacy, as a formal privacy protection method, typically limits the impact of individual client data on model output by injecting random noise into model updates, and has been applied in traditional federated learning scenarios. However, in the federated LoRA fine-tuning architecture, model updates consist of low-rank matrix multiplication. If noise is directly injected into the low-rank factors, multiplicative interactions will occur during weight reconstruction, forming secondary noise terms, leading to noise amplification and severely affecting model performance and training stability.

[0006] The following technical problems exist in existing federal LoRA differential privacy methods:

[0007] 1. When differential privacy noise is directly introduced under the federated LoRA architecture, due to the low-rank matrix multiplication structure, a quadratic noise term will be generated, leading to noise amplification;

[0008] 2. Existing differential privacy federated LoRA methods typically do not distinguish the functional differences between the directional and magnitude components in the LoRA parameters. This can easily lead to overall perturbation of the A and B matrices or their decomposition subcomponents, causing key shared knowledge components or personalized components to be affected by unnecessary noise, thereby reducing fine-tuning efficiency and model performance.

[0009] Currently, there is a lack of an effective technical solution that can simultaneously balance the fine-tuning efficiency of federated LoRA, the strength of differential privacy protection, and the suppression of noise amplification. Therefore, it is necessary to propose a new differential privacy noise injection method for direction-amplitude decomposition federated LoRA to address the aforementioned issues. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides an efficiency-aware differential privacy noise injection method for fine-tuning of direction-amplitude decomposition-based federated LoRA. This method is suitable for efficient fine-tuning of large model parameters in federated learning environments, particularly for LoRA-based federated fine-tuning scenarios. Instead of directly adding noise to ordinary LoRA modules, this method performs reparameterization training and differential privacy noise injection on the target decomposition subcomponents of the LoRA adaptation parameters within the direction-amplitude decomposition-based federated LoRA optimization architecture. The LoRA module can be implemented using existing publicly available Low-Rank Adaptation structures. The key improvement of this invention lies in the parameter pruning, hierarchical perturbation, noise reconstruction, and federated aggregation processes within this decomposition-based federated LoRA architecture, achieving a balance between privacy protection, model performance, and training efficiency.

[0011] On one hand, the present invention provides an efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning, comprising the following steps:

[0012] Step 1: During each round of communication, the client performs a local training phase for reparameterization: by freezing some LoRA factors and alternately updating the remaining LoRA factors, only the low-rank components are optimized;

[0013] Specifically, the LoRA module is represented as the product of a low-rank matrix A and a low-rank matrix B, where the low-rank matrix is ​​further decomposed into directional and magnitude components; specifically, both the low-rank matrix A and the low-rank matrix B are decomposed into the product of the directional matrix and the magnitude matrix.

[0014] The local training phase for reparameterization specifically includes:

[0015] 1. Directional component training phase: Only the directional components of the low-rank matrix A are optimized, and amplitude-related parameters are frozen;

[0016] 2. Amplitude component training phase: Only the amplitude components of the low-rank matrix B are optimized, and the direction-related parameters are frozen;

[0017] Step 2: Perform local importance assessment for LoRA updates across layers; first, calculate the importance score of each LoRA layer based on the LoRA factor norm, and then divide the different layers into several levels with different importance levels accordingly;

[0018] Specifically, the Frobenius norm of the corresponding low-rank matrix of each LoRA layer is calculated, and the norm is used as the importance score of the parameters of the LoRA layer. Based on the importance score, all LoRA layers are divided into three importance levels according to the proportions of 30%, 40%, and 30%, from high to low: important layers, general layers, and unimportant layers.

[0019] Step 3: The client prunes the LoRA parameter updates to be uploaded; after completing the importance assessment of the LoRA layer, the client prunes the LoRA parameter updates obtained from this round of local training to limit the magnitude of individual client parameter updates and provide sensitivity constraints for subsequent differential privacy noise injection.

[0020] Specifically, the client first calculates the overall norm of the LoRA parameter update to be uploaded and compares it with a preset pruning threshold C. If the norm of the parameter update to be uploaded does not exceed the pruning threshold C, the parameter update remains unchanged. If the norm of the parameter update to be uploaded exceeds the pruning threshold C, the parameter update is scaled proportionally to ensure that its norm does not exceed the pruning threshold C. After the above pruning, the client obtains the pruned LoRA parameter update and uses it as input for subsequent hierarchical perturbation strategy determination and noise reconstruction and injection.

[0021] Step 4: Introduce a hierarchical perturbation mechanism; based on the importance level classification, adopt a differentiated noise injection strategy for LoRA layers of different importance levels; specifically, for important layers, skip differential privacy noise injection; for general layers, trigger noise injection with a preset 50% probability; for unimportant layers, perform noise injection, so that different levels receive differentiated processing during the perturbation injection process.

[0022] Among them, for low-rank matrices A and B within the same LoRA layer, their perturbation strategy is uniformly determined by the importance level of the LoRA layer. Based on the importance level of the LoRA layer, differential noise injection is performed on the corresponding sub-components obtained after the direction-magnitude decomposition of the low-rank matrix.

[0023] Step 5: The client performs noise reconstruction surgery; under the premise of satisfying differential privacy constraints, the redundant noise terms introduced by the low-rank decomposition are eliminated from the structural level by deterministically mapping the injection method of the original noise in the weight space back to the LoRA factor space.

[0024] Specifically, noise will be uniformly generated in the equivalent weight update space, and a deterministic mapping relationship will be constructed based on the reparameterized structure of the current LoRA module to reconstruct the noise into an equivalent perturbation acting on the specified sub-component.

[0025] Step 5.1: Suppose that the noise matrix η is uniformly sampled once in the equivalent weight update space, and the reparameterized representation A=D based on the current LoRA module is calculated. A M A B=D B M B ; where D A M A These are the direction matrix and magnitude matrix of the low-rank matrix A, respectively, and D. B M B These are the direction matrix and magnitude matrix of the low-rank matrix B, respectively;

[0026] Construct a deterministic noise reconstruction map to reconstruct η into values ​​acting on the direction matrix D. A With amplitude matrix M B noise item and The final reconstructed noise satisfies the following expression:

[0027] ;

[0028] Step 5.2: Inject noise into the parameters; reconstruct the noise. Injected into direction matrix D A middle: ;

[0029] Then there is

[0030] in, Indicates that the noise will be reconstructed. Write back to direction matrix D A The resulting noise direction matrix; This represents the equivalent weights when injecting differential privacy noise, where W0 is the frozen base model weights; the above-constructed pair Noise reconstruction mapping, has ;

[0031] Therefore, we get: ;

[0032] Reconstruction noise Injected into M B middle: ;

[0033] in, Indicates that the noise will be reconstructed. Write back to magnitude matrix M B The resulting noise amplitude matrix;

[0034] but: ;

[0035] Through the above-constructed pair Noise reconstruction mapping, has ;

[0036] Thus obtain ;

[0037] Step 6: The LoRA parameter updates after noise reconstruction are uploaded to the server for federated aggregation and broadcast back to each client as initialization parameters for the next round of local training.

[0038] On the other hand, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning.

[0039] The beneficial effects of adopting the above technical solution are as follows:

[0040] This invention provides an efficiency-aware differential privacy noise injection method for direction-amplitude decomposition-based federated LoRA fine-tuning, which effectively solves the problems of noise amplification and performance degradation in existing federated LoRA differential privacy fine-tuning methods, and has good engineering practical value and application prospects. Compared with the prior art, this invention has at least the following beneficial effects:

[0041] 1. Through noise reconstruction surgery, the secondary noise term caused by the low-rank product structure in the direction-amplitude decomposition federated LoRA fine-tuning is effectively eliminated, and noise amplification is significantly suppressed;

[0042] 2. Through a graded perturbation mechanism, a structured allocation of the privacy budget was achieved, reducing unnecessary perturbations to key parameters;

[0043] 3. While satisfying the standard difference privacy constraint, the stability and final accuracy of model fine-tuning are improved;

[0044] 4. Experimental results on multiple models and tasks show that the method of this invention improves the global model accuracy by 1.75% compared with the existing baseline method. Attached Figure Description

[0045] Figure 1 This is an overall structural diagram of the efficiency-aware differential privacy noise injection method according to an embodiment of the present invention;

[0046] Figure 2 This illustrates the effect of the noise factor on the noise level in an embodiment of the present invention. Detailed Implementation

[0047] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0048] Example 1:

[0049] On one hand, this invention provides an efficiency-aware differential privacy noise injection method for direction-amplitude decomposition-based federated LoRA fine-tuning. This method is designed around the overall training process of direction-amplitude decomposition-based federated LoRA fine-tuning and is implemented by multiple mutually cooperating and tightly coupled functional modules, such as... Figure 1 As shown, it includes the following steps:

[0050] Step 1: During each round of communication, the client performs a local training phase with reparameterization: by freezing some LoRA factors and alternately updating the remaining LoRA factors, only the low-rank components are optimized, thereby significantly reducing communication and computational overhead while effectively maintaining the expressive power of the LoRA structure.

[0051] Specifically, the LoRA module is represented as the product of a low-rank matrix A and a low-rank matrix B, where the low-rank matrix is ​​further decomposed into directional and magnitude components; specifically, both the low-rank matrix A and the low-rank matrix B are decomposed into the product of the directional matrix and the magnitude matrix.

[0052] The local training phase for reparameterization specifically includes:

[0053] 1. Directional component training phase: Only the directional components of the low-rank matrix A are optimized, and amplitude-related parameters are frozen;

[0054] 2. Amplitude component training phase: Only the amplitude components of the low-rank matrix B are optimized, and the direction-related parameters are frozen;

[0055] By using the aforementioned phased training method, we can significantly reduce the number of trainable parameters while maintaining model expressiveness similar to that of traditional LoRA fine-tuning, and provide a clear parameter subspace partitioning basis for subsequent structured injection of differential privacy noise.

[0056] Step 2: Perform local importance assessment for LoRA updates across layers; first, calculate the importance score of each LoRA layer based on the LoRA factor norm, and then divide the different layers into several levels with different importance levels accordingly;

[0057] Specifically, the Frobenius norm of the corresponding low-rank matrix of each LoRA layer is calculated, and the norm is used as the importance score of the parameters of the LoRA layer. Based on the importance score, all LoRA layers are divided into three importance levels according to the proportions of 30%, 40%, and 30%, from high to low: important layers, general layers, and unimportant layers.

[0058] Step 3: The client prunes the LoRA parameter updates to be uploaded; after completing the importance assessment of the LoRA layer, the client prunes the LoRA parameter updates obtained from this round of local training to limit the magnitude of individual client parameter updates and provide sensitivity constraints for subsequent differential privacy noise injection.

[0059] Specifically, the client first calculates the overall norm of the LoRA parameter update to be uploaded and compares it with a preset pruning threshold C. If the norm of the parameter update to be uploaded does not exceed the pruning threshold C, the parameter update remains unchanged. If the norm of the parameter update to be uploaded exceeds the pruning threshold C, the parameter update is scaled proportionally to ensure that its norm does not exceed the pruning threshold C. After the above pruning, the client obtains the pruned LoRA parameter update and uses it as input for subsequent hierarchical perturbation strategy determination and noise reconstruction and injection.

[0060] Step 4: Introduce a hierarchical perturbation mechanism; based on the importance level classification, adopt a differentiated noise injection strategy for LoRA layers of different importance levels; specifically, for important layers, skip differential privacy noise injection; for general layers, trigger noise injection with a preset 50% probability; for unimportant layers, perform noise injection, so that different levels receive differentiated processing during the perturbation injection process, thereby effectively avoiding performance degradation of critical layers due to excessive noise.

[0061] Furthermore, based on the hierarchical division completed in step 2 according to the importance scores of each LoRA layer, step 4 directly uses the results of this hierarchical division to determine the differentiated perturbation strategy. For low-rank matrices A and B within the same LoRA layer, their perturbation strategies are uniformly determined by the overall importance level of that LoRA layer, without independently determining the level of low-rank matrices A and B separately. It should be noted that low-rank matrices A and B are not recalculated for importance levels as new scoring objects in this step, and noise is not directly injected into low-rank matrices A and B. Instead, differentiated noise injection is performed on the corresponding sub-components obtained after direction-amplitude decomposition of the low-rank matrices according to the importance level of the LoRA layer. Through the above method, the inconsistency of perturbation strategies caused by independently determining the levels of paired low-rank matrices within the same LoRA layer can be avoided, and the privacy budget can be structurally allocated in the decomposed LoRA parameter space. This hierarchical perturbation method achieves the structural allocation of the privacy budget in the parameter space without increasing the additional communication and computational burden.

[0062] Step 5: The client performs noise reconstruction surgery; under the premise of satisfying differential privacy constraints, the original noise is deterministically mapped back to the LoRA factor space by injecting the original noise into the weight space, thereby eliminating redundant noise terms introduced by low-rank decomposition at the structural level, and thus improving the effectiveness and stability of model training under privacy protection.

[0063] Specifically, noise will be uniformly generated in the equivalent weight update space, and a deterministic mapping relationship will be constructed based on the reparameterized structure of the current LoRA module to reconstruct the noise into an equivalent perturbation acting on the specified sub-component.

[0064] To avoid the secondary noise term introduced by independently adding noise to multiple sub-components under the low-rank product structure of LoRA, this invention proposes a noise reconstruction procedure. In this embodiment, instead of sampling noise separately on each LoRA sub-component, noise is uniformly generated in the equivalent weight update space. Based on the reparameterized structure of the current LoRA module, a deterministic mapping relationship is constructed, and the noise is reconstructed into an equivalent perturbation acting on the specified sub-component. This allows the noise to propagate linearly during parameter combination, thereby eliminating the secondary noise term mathematically.

[0065] Step 5.1: Suppose that the noise matrix η is uniformly sampled once in the equivalent weight update space, and the reparameterized representation A=D based on the current LoRA module is calculated. A M A B=D B M B ; where D A M AThese are the direction matrix and magnitude matrix of the low-rank matrix A, respectively, and D. B M B These are the direction matrix and magnitude matrix of the low-rank matrix B, respectively;

[0066] Construct a deterministic noise reconstruction map to reconstruct η into values ​​acting on the direction matrix D. A With amplitude matrix M B noise item and The final reconstructed noise satisfies the following expression:

[0067] ;

[0068] It should be noted that in actual calculations of the noise reconstruction map, if the correlation matrix does not satisfy the full-rank condition or exhibits ill-conditioned behavior, direct inversion may lead to numerical instability. To address this issue, this embodiment uses the Tikhonov-regularized generalized inverse for alternative calculation. Specifically, the inversion operation... Equivalent replacement to regularized generalized inverse , where λ is a small regularization coefficient. This process can improve computational stability in cases where the matrix is ​​non-invertible or approximately singular while maintaining the determinism of the noisy reconstruction mapping. The above construction , The formula is mainly used to illustrate the basic idea of ​​noise reconstruction mapping, making it easier to understand its equivalent solution process. In practical implementation, the Tikhonov regularized generalized inverse method is used to achieve a stable solution. Therefore, this stable solution form is adopted in subsequent experiments and algorithm implementation to reduce the influence of the ill-conditioned matrix on the noise reconstruction results.

[0069] Step 5.2: Inject noise into the parameters; reconstruct the noise. Injected into direction matrix D A middle: ;

[0070] Then there is

[0071] in, Indicates that the noise will be reconstructed. Write back to direction matrix D A The resulting noise direction matrix; This represents the equivalent weights when injecting differential privacy noise, where W0 is the frozen base model weights; the above-constructed pair Noise reconstruction mapping, has ;

[0072] Therefore, we get: ;

[0073] Reconstruction noise Injected into M B middle: ;

[0074] in, Indicates that the noise will be reconstructed. Write back to magnitude matrix M B The resulting noise amplitude matrix;

[0075] but: ;

[0076] Through the above-constructed pair Noise reconstruction mapping, has ;

[0077] Thus obtain ;

[0078] In this embodiment, as a comparative verification, if there is no reconstruction noise, a secondary noise term will appear. The specific derivation steps are as follows:

[0079] (1) The equivalent weight update of LoRA is expressed as: = +BA= + ;

[0080] (2) Noise Injected into middle: ;

[0081] Then there is ;

[0082] (3) Noise Injected into M B middle: ;

[0083] ;

[0084] The last item For the quadratic noise term caused by the product structure, previous studies have shown that when quadratic noise occurs, the noise norm it generates is much larger than the norm of the first-order noise term as the noise intensity increases. Figure 2 This trend is further demonstrated: as the noise factor increases, the combined noise... The intensity compared to single noise or The intensity showed a faster growth trend. Figure 2 In the middle, the noise factor on the horizontal axis The scaling factor for Gaussian noise in differential privacy training is used to control the intensity of injected noise; the noise magnitude on the ordinate represents the overall perturbation of the noise matrix, which is quantified using the Frobenius norm. As the noise level increases, the intensity of the injected noise gradually increases, and the norm of the corresponding noise matrix also increases. Compared to directly adding noise to the full parameter matrix, the combined noise product term in the LoRA decomposition structure... Compared to single noise terms It exhibits a faster growth trend.

[0085] As can be seen from the above derivation, the present invention generates noise uniformly in the equivalent weight update space and uses deterministic mapping to write back to the parameter subcomponents of the reparameterized LoRA, so that the disturbance term enters the weight update expression only in a linear form, and there is no quadratic noise term generated by multiplying multiple independent noises, thereby eliminating the noise amplification phenomenon in the mathematical structure.

[0086] Step 6: The LoRA parameter updates after noise reconstruction are uploaded to the server for federated aggregation and broadcast back to each client as the initialization parameters for the next round of local training.

[0087] After completing the reparameterization training, importance assessment, parameter pruning, hierarchical perturbation, and noise reconstruction processes, the client uploads the processed LoRA parameter updates to the server. In this embodiment, the server uses a weighted average federated aggregation method, summing the parameter updates from each client based on their local data volume to obtain new global LoRA parameters. The server does not perform any operations related to differential privacy, nor does it access any client's original data, thus ensuring the overall system's privacy and security while maintaining its simplicity.

[0088] Example 2:

[0089] Models: We evaluated the models on two representative large language models, LLaMA2-7B and ChatGLM2-6B.

[0090] Task: We selected three downstream tasks from the Lawyer-Instruct and Natural Instruction datasets: law, philosophy, and physics. Accuracy was used as the evaluation metric in the experiments. The test set consisted of multiple-choice questions, requiring the model to output a unique answer from given options. Accuracy was used as the evaluation metric, and the exact-match rule was applied: the model was considered correct only if its output perfectly matched the standard answer; otherwise, it was considered incorrect.

[0091] Federated Learning and Privacy Settings: In the federated learning settings, the number of clients is set to 10. To protect client data privacy, we employ a differential privacy mechanism and set the privacy budget to ε = 6 and δ = 10. -3 Here, ε is the privacy budget in differential privacy, used to measure the strength of privacy protection. Generally, the smaller ε is, the stronger the privacy protection, but the model usability may decrease; the larger ε is, the weaker the added perturbation, and the model performance may be better, but the privacy protection is relatively weakened; δ represents the upper bound of the probability of privacy protection failure, used to describe the extremely small probability of privacy constraints being broken. This paper further sets the noise factor to σ=1.0, the gradient pruning threshold to C=1.0, the client sampling rate to q=0.5, and the number of communication rounds to T=100, and uses the RDP privacy accounting method to accumulate privacy loss in multiple rounds of communication. Here, the noise factor σ is used to control the scaling magnitude of Gaussian noise, the larger σ is, the stronger the added noise; the gradient pruning threshold C is used to limit the norm of parameter updates uploaded by the client to avoid excessive update magnitude by a single client; the client sampling rate q represents the proportion of clients participating in training in each round; the number of communication rounds T represents the total number of interactions between the client and the server; the RDP privacy accounting method is used to accumulate privacy loss in multiple rounds of training and determine whether it meets the given ε and δ privacy constraints. The parameters mentioned above collectively determine the noise injection intensity under the differential privacy mechanism. During each round of communication, the client updates the model using local data and only uploads the noise-perturbed parameter updates to the server.

[0092] LoRA Fine-tuning Settings: During the efficient parameter fine-tuning phase, a direction-amplitude decomposition federated LoRA optimization architecture is used to fine-tune the backbone model. The LoRA rank is set to 16, and the optimizer learning rate is set to 2 × 10⁻⁶. -5 The batch size for single-card training is set to 2, and the gradient accumulation step count is set to 4 to maintain training stability under limited GPU memory conditions.

[0093] Example 3:

[0094] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning.

[0095] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0096] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0097] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. An efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning, characterized in that, Includes the following steps: Step 1: During each round of communication, the client performs a local training phase for reparameterization: by freezing some LoRA factors and alternately updating the remaining LoRA factors, only the low-rank components are optimized; Step 2: Perform local importance assessment for LoRA updates across layers; first, calculate the importance score of each LoRA layer based on the LoRA factor norm, and then divide the different layers into several levels with different importance levels accordingly; Step 3: The client prunes the LoRA parameter updates to be uploaded; After completing the importance assessment of the LoRA layer, the client prunes the LoRA parameter updates obtained from the local training in this round to limit the magnitude of parameter updates by a single client and to provide sensitivity constraints for subsequent differential privacy noise injection. Step 4: Introduce a hierarchical perturbation mechanism; based on the importance level classification results, adopt differentiated noise injection strategies for LoRA layers of different importance levels; Specifically, for important layers, differential privacy noise injection is skipped; for general layers, noise injection is triggered with a preset probability of 50%. For unimportant layers, noise injection is performed to ensure that different layers receive differentiated treatment during the disturbance injection process. Step 5: The client performs noise reconstruction surgery; under the premise of satisfying differential privacy constraints, the redundant noise terms introduced by the low-rank decomposition are eliminated from the structural level by deterministically mapping the injection method of the original noise in the weight space back to the LoRA factor space. Step 6: The LoRA parameter updates after noise reconstruction are uploaded to the server for federated aggregation and broadcast back to each client as initialization parameters for the next round of local training.

2. The efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning according to claim 1, characterized in that, Step 1 specifically involves representing the LoRA module as a product of a low-rank matrix A and a low-rank matrix B, wherein the low-rank matrix is ​​further decomposed into directional components and magnitude components; specifically, both the low-rank matrix A and the low-rank matrix B are decomposed into a product of a directional matrix and a magnitude matrix.

3. The efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning according to claim 2, characterized in that, The reparameterization local training phase specifically includes: (1) Directional component training stage: Only the directional components of the low-rank matrix A are optimized, and the amplitude-related parameters are frozen; (2) Amplitude component training stage: Only the amplitude components of the low-rank matrix B are optimized, and the direction-related parameters are frozen.

4. The method of claim 1, wherein, Step 2 specifically involves: calculating the Frobenius norm of the corresponding low-rank matrix for each LoRA layer, and using this norm as the importance score of the LoRA layer parameters; based on the importance score results from high to low, dividing all LoRA layers into three importance levels according to the proportions of 30%, 40%, and 30%, including important layers, general layers, and unimportant layers.

5. The method of claim 1, wherein, Specifically, step 3 involves the client first calculating the overall norm of the LoRA parameter update to be uploaded and comparing it with a preset pruning threshold C; if the norm of the parameter update to be uploaded does not exceed the pruning threshold C, then the parameter update remains unchanged. If the norm of the parameter update to be uploaded exceeds the pruning threshold C, the parameter update is scaled proportionally so that its norm does not exceed the pruning threshold C. After the above pruning, the client obtains the pruned LoRA parameter update and uses it as the input for subsequent hierarchical perturbation strategy determination and noise reconstruction and injection.

6. The method of claim 1, wherein, In step 4, for low-rank matrices A and B within the same LoRA layer, the perturbation strategy is uniformly determined by the importance level of the LoRA layer. Based on the importance level of the LoRA layer, differential noise injection is performed on the corresponding sub-components obtained after direction-amplitude decomposition of the low-rank matrix.

7. The method of claim 1, wherein, Specifically, step 5 will generate noise uniformly in the equivalent weight update space, and construct a deterministic mapping relationship based on the reparameterized structure of the current LoRA module, and reconstruct the noise into an equivalent disturbance acting on the specified sub-component.

8. The efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning according to claim 7, characterized in that, Step 5 specifically includes the following steps: Step 5.1: Suppose that the noise matrix η is uniformly sampled once in the equivalent weight update space, and the reparameterized representation A=D based on the current LoRA module is calculated. A M A B=D B M B ; where D A M A These are the direction matrix and magnitude matrix of the low-rank matrix A, respectively, and D. B M B These are the direction matrix and magnitude matrix of the low-rank matrix B, respectively; Construct a deterministic noise reconstruction map to reconstruct η into values ​​acting on the direction matrix D. A With amplitude matrix M B noise item and The final reconstructed noise satisfies the following expression: ； Step 5.2: Inject noise into the parameters; reconstruct the noise. Injected into direction matrix D A middle: ; Then there is ; in, Indicates that the noise will be reconstructed. Write back to direction matrix D A The resulting noise direction matrix; This represents the equivalent weights when injecting differential privacy noise, where W0 is the frozen base model weights; the above-constructed pair Noise reconstruction mapping, has ; Therefore, we can conclude that: ; Reconstruction noise Injected into M B middle: ; in, Indicates that the noise will be reconstructed. Write back to magnitude matrix M B The resulting matrix of noise amplitude values; but: ; Through the above-constructed pair Noise reconstruction mapping, has ; Thus obtain .

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the efficiency-aware differential privacy noise injection method for direction-amplitude decomposition federated LoRA fine-tuning as described in any one of claims 1-8.

Images