Method and system for building heterogeneous federated fine-tuning language models based on differential privacy

Abstract

The present invention discloses a method and system for constructing a heterogeneous federated fine-tuning language model based on differential privacy. The parameter server first clusters clients with similar capabilities, clusters them based on resource awareness, and obtains a larger rank adaptive matrix for each class according to the category with large computing resources and computing power, thereby achieving a flexible low-rank adaptive matrix rank to enhance fine-tuning performance. Within each class, privacy federated fine-tuning is adopted, including two parts: noise allocation with sensitivity awareness and regularization term optimization, to achieve better training efficiency and reduced noise impact. Between each class, heterogeneous low-rank adaptive matrix aggregation is performed through knowledge distillation technology, which includes knowledge aggregation and selection of high-quality knowledge to enhance local models. The present invention can provide strict privacy protection, and can alleviate the impact of differential privacy on low-rank adaptive matrix fine-tuning in a heterogeneous environment, achieving higher efficiency and accuracy.

Description

Technical Field

[0001] The present invention belongs to the fields of computer network security technology and cryptography technology, and relates to a method and system for constructing an efficient heterogeneous federated fine-tuning language model. Specifically, it is designed to construct a heterogeneous federated fine-tuning method and system based on sensitivity-aware differential privacy noise distribution. Background Art

[0002] Pre-trained language models are first trained on large public datasets and then fine-tuned on domain-specific private datasets. They have demonstrated great potential in many scenarios. However, due to the large number of parameters, directly fine-tuning these pre-trained models incurs high computational overhead. LoRA low-rank adaptive technology, as a parameter-efficient fine-tuning method, can achieve similar performance to fine-tuning all parameters while fine-tuning a small number of parameters. However, LoRA also requires the collection of rich data for fine-tuning, which undoubtedly introduces privacy risks. Federated low-rank adaptive fine-tuning allows multiple clients to collaboratively fine-tune pre-trained language models on downstream tasks without exposing private data. However, this paradigm has two inherent issues. First, due to the heterogeneity of client capabilities and data distribution, model performance can be degraded, significantly reducing the efficiency of model fine-tuning. Second, due to shared model gradients, sending model parameters or gradients to a parameter server for aggregation poses the risk of privacy leakage, which has been proven to be unsafe in many studies. Although differential privacy provides a solution to the privacy leakage problem by adding random noise to the transmitted parameters or model updates, theoretically ensuring that the adversary cannot perform inference attacks, this privacy protection technology will sacrifice model performance due to random noise, and this will be more significant under low-rank adaptive fine-tuning. Fine-tuning a small number of parameters is very sensitive to changes in the model, which will amplify the negative impact of differential privacy noise on accuracy.

[0003] Current federated fine-tuning for differential privacy protection still faces several challenges. The values ​​of low-rank adaptive parameters vary, resulting in varying sensitivities to added noise. Existing differential privacy schemes, however, apply a uniform noise level to all parameters. Secondly, local models are trained on non-IID datasets, and low-rank adaptive matrices of different ranks will perform differently on these datasets. Low-rank models are more sensitive due to their lower generalization ability. Furthermore, existing federated low-rank adaptive fine-tuning approaches simply combine parameter-efficient methods with federated learning, ignoring the limited and heterogeneous capabilities of real-world clients. Existing methods achieve aggregation of heterogeneous low-rank adaptive matrices through methods such as singular value decomposition, but their performance still needs improvement. Therefore, designing a flexible and efficient differential privacy protection scheme for federated fine-tuning of heterogeneous low-rank adaptive matrices, and allocating adaptive matrices of a rank that adapts to client resource capabilities, remains an urgent challenge. Summary of the Invention

[0004] In order to solve the technical problems of low-rank adaptive matrix noise sensitivity and client heterogeneity, the present invention provides a sensitivity-aware differential privacy to implement a flexible and efficient heterogeneous federated fine-tuning system construction method and system.

[0005] The technical solution adopted by the method of the present invention is: a method for constructing a heterogeneous federated fine-tuning language model based on differential privacy; the heterogeneous federated fine-tuning language model, whose client local model is obtained through training; the training process specifically includes the following steps:

[0006] Step 1: The parameter server clusters all participating N clients into C categories based on the resource capabilities of each client, and assigns a low-rank adaptive matrix LoRA of corresponding level (the level corresponds to the size of the rank) to each category;

[0007] Step 2: At the beginning of each round of training of the client local model, the parameter server selects the participating client set P for each class c c and a clustering client p c , each selected client trains and updates the local model multiple times; the selected client participates in this round of training, performs low-rank adaptive matrix LoRA fine-tuning on the local model on its own private dataset, and only updates the local trainable parameters w i , freeze the pre-training weight W0;

[0008] Step 3: After the selected client completes the local model update, it performs privacy fine-tuning, adds parameter-sensitive differential privacy noise and regularization term optimization to the local model update, and sends the privacy-preserving privacy model update to the cluster client PC;

[0009] Step 4: Perform homogeneous aggregation within the class; for each class c, the clustering client pc of each class receives the information of other clients i∈P in the same class. c Privacy model update in Include only LoRA trainable parameters, excluding the parameters of the frozen part, and perform federated average aggregation of FedAvg to obtain the clustering model w c =[A c , B c ]; among them, A c , B c are respectively the aggregated w c The corresponding two low-rank matrices (LoRA is a combination of two low-rank matrices A and B);

[0010] Step 5: Perform heterogeneous aggregation between classes; the clustering client of each class is in the public dataset Computational knowledge c and sends the knowledge to the parameter server for subsequent heterogeneous LoRA aggregation;

[0011] Step 6: The parameter server first calculates the aggregation weight ω based on the correlation between the knowledge received for each class. c , then perform weight aggregation to obtain global knowledge f;

[0012] Step 7: The parameter server selects the knowledge f whose confidence of each public dataset sample is greater than the threshold φ from the aggregated knowledge s , and sent to each clustering client;

[0013] Step 8: Each cluster client p c , c≤C for the received global selected knowledge f s , perform the knowledge distillation process on the public dataset and update the clustering model w c , and sent to each client P in the class c ;

[0014] Step 9: The clients in each cluster receive the cluster client p c The clustering model w issued c Parameters, update your local LoRA module w i and the local model W i =W0+B c A c .

[0015] As a preference, in step 1, the parameter server first performs resource-aware clustering on all clients and defines the resource feature vector χ i =[χ i，1 , χ i，2 , χi，3 , χ i，4 ], where χ i，1 represents the data volume feature, χ i，2 represents the computing capability characteristics, χ i3 Represents the storage capacity characteristics, χ i，4 Represents the communication capability feature; for features of different dimensions, first normalize them using Min-Max, then randomly select C cluster centers, and calculate the fuzzy membership of each client i and the cluster center c represents the probability that the client belongs to cluster c, d(χ i , v c ) represents the distance between client i and cluster center v c distance, m is a fuzzy factor greater than 1; then client i is assigned to the cluster with the highest membership; then the cluster center c is updated for each class, and the above process is repeated until the cluster center no longer changes.

[0016] As a preference, in step 3, after the local fine-tuning is completed, the client updates the local model by Δw i Each element of adds sensitivity-aware differential privacy, and each value adds a noise size related to its own sensitivity.

[0017] Preferably, the sensitivity-aware differential privacy is achieved by first estimating the corresponding clipping threshold s for each client i’s m-th local model update. i，m , and then estimate the sensitivity of the model update And calculate the noise level corresponding to each model update where M is the number of parameter updates, σ * is the overall noise level.

[0018] Preferably, in step 6, the parameter server calculates the relevance of the knowledge sent by each cluster client, where the Pearson correlation coefficient is used as the weight measurement indicator, and the average knowledge of all clusters is used as a reference. The greater the correlation, the greater the aggregation weight assigned.

[0019] Preferably, the Pearson correlation coefficient in, From public datasets Sample Computed knowledge, is the average clustering knowledge of all samples, Indicates that for the sample The average knowledge of all C clusters; the average knowledge of all classes And the correlation coefficient ρ cNormalization is used as the weight of knowledge aggregation; after that, the parameter server normalizes the correlation coefficient. Calculate the weight ω c After that, perform weight aggregation on the knowledge

[0020] As a preference, in step 7, the parameter server calculates the confidence of each aggregated knowledge in Represents a sample The corresponding knowledge Represents the probability distribution after probability normalization; when the confidence φi>φ, where φ is the confidence threshold.

[0021] As a preference, in step 8, each clustering client pc receives the selected global knowledge f fed back by the parameter server s , perform the knowledge distillation process, the loss function is in is f s Corresponding to the selected data set, ξ is the adjustment factor, τ is the distillation temperature, and σ is the activation function; Represents a sample, represents the sample, W c are the model parameters of cluster c, is the cross entropy loss function, is the global knowledge of server selection, Is cluster c in the sample Knowledge on D KL () represents the KL divergence loss function.

[0022] The technical solution adopted by the system of the present invention is: a heterogeneous federated fine-tuning language model construction system based on differential privacy, including:

[0023] one or more processors;

[0024] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the heterogeneous federated fine-tuning language model construction method based on differential privacy.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) The present invention provides an efficient differentially private heterogeneous federated fine-tuning language model (DP-FedLoRA). The existing technology does not fully consider the differences in client resource capabilities, ignores the different sensitivity of low-rank adaptive matrices to noise, and ignores the impact of non-independent and identically distributed data on federated fine-tuning performance. The present invention integrates resource-aware client clustering, intra-class privacy federated fine-tuning, and aggregation of heterogeneous low-rank adaptive matrices between classes for the first time, solving the challenges brought by noise and heterogeneity in the actual use of federated devices, and achieving both privacy protection and efficiency.

[0027] (2) The present invention proposes an efficient, resource-aware clustering method. The parameter server clusters clients according to their different resource levels and assigns clients with similar resource levels to the same class. The existing technology usually clusters based on data similarity, but it is difficult to ensure data privacy while evaluating data similarity. Moreover, data similarity clustering only considers the resource feature of data, ignoring important resource features such as computing power, communication bandwidth, and storage capacity in the federated fine-tuning process. The present invention comprehensively considers different resource attributes and combines them into resource vectors. During clustering, fuzzy membership is used to approximate the probability that the client belongs to the current class, and the client is assigned to the class with the highest probability. Such a clustering method is also much more efficient than the usual calculation of cosine similarity.

[0028] (3) The intra-class regularization optimization proposed in the present invention is used to limit the update direction and step size of the local model so as not to deviate too much from the model of the current class, so as to alleviate the aggregation bias caused by non-independent and identically distributed data, and the restriction of the regularization term is combined when the local model calculates the loss function. At the same time, the regularization term of the present invention is based on the rank size of each class. For smaller ranks, the generalization ability of the low-rank adaptive matrix is ​​poor, and more restrictions on the regularization term are required. For larger ranks, the restriction of the regularization term can be reduced, and the model is less affected by non-independent and identically distributed data. The problem of performance degradation under non-independent and identically distributed conditions is alleviated, and rank-related regularization term restrictions are added for different ranks to give full play to the generalization ability of higher-rank models.

[0029] (4) The present invention adds sensitivity-aware noise to each model update, estimates the local model update through past gradient updates, and uses this as an estimate of the clipping threshold for the model update. The noise level corresponding to each model update is then calculated, and the adaptive noise is added at a scalar granularity. The present invention adds sensitivity-aware noise to each element of the model update. Model updates with less sensitivity do not require excessive noise protection, while model updates with greater sensitivity add greater noise to protect privacy. The noise distribution scheme of the present invention can significantly mitigate the degradation of model performance caused by noise while maintaining the same level of privacy protection, especially for low-rank adaptive matrices.

[0030] (5) The present invention provides a knowledge distillation method for the aggregation of heterogeneous low-rank adaptive matrices. Existing technologies usually solve the problem of heterogeneous aggregation by cutting or filling, but cannot give full play to the performance of heterogeneous aggregation. The present invention transmits knowledge of different classes between classes, and the parameter server performs weighted aggregation on the knowledge and calculates the Pearson correlation coefficient between the knowledge as the aggregation weight of the knowledge. The knowledge with higher correlation has higher consistency and is given a higher aggregation weight. The aggregated knowledge is selected through a knowledge selection scheme to select some high-quality knowledge, that is, knowledge with higher confidence. The global client has high confidence in this type of knowledge and can make accurate predictions. The selected knowledge is fed back to each class for the knowledge distillation process. In addition, knowledge distillation, as a smoothing technology, can also better handle the problem of non-independent and identically distributed data, and realizes efficient knowledge migration between clients with different model structures between classes, clustering clients, and parameter servers, and provides higher heterogeneous aggregation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The technical solution of this invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without inventive efforts.

[0032] Figure 1 This is a diagram showing the principle of training a heterogeneous federated fine-tuning language model with differential privacy according to an embodiment of the present invention;

[0033] Figure 2 This is a flow chart of resource-aware clustering according to an embodiment of the present invention;

[0034] Figure 3 This is a flow chart of heterogeneous aggregation within a class and knowledge migration between classes according to an embodiment of the present invention;

[0035] Figure 4This is a flowchart of sensitivity-aware differential privacy noise allocation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0037] Federated fine-tuning of language models enables multiple clients to collaboratively fine-tune a pre-trained language model without sharing local data. To prevent privacy leaks during the aggregation process, differential privacy is widely used in federated learning. Local model updates are pruned and noise is added, but uniform random noise can lead to significant performance losses. Furthermore, to fully leverage the diverse resource capabilities of clients, heterogeneous low-rank adaptive modules are used across heterogeneous clients in federated learning. However, efficiently aggregating heterogeneous fine-tunable modules while maintaining model performance is a major challenge.

[0038] This embodiment proposes an efficient differential privacy heterogeneous federated fine-tuning language model, such as Figure 1 As shown, compatible resource-aware client clustering, rank-related regularization optimization, sensitivity-aware noise distribution, and inter-class weight knowledge distillation and knowledge selection are used to solve performance degradation and heterogeneity problems. The technologies involved in this embodiment include resource-aware client clustering, regularization optimization, sensitivity-aware differential privacy technology, correlation-based weighted knowledge aggregation, and confidence-based knowledge selection mechanism.

[0039] First, the formal definition of the t-th round symbol of this system is given:

[0040] Knowledge distillation loss function

[0041] Client set {i|i=1, 2, 3, ..., N};

[0042] Client parameters

[0043] Client-side gradient updates

[0044] Client-side clipped gradient updates

[0045] Client-side gradient update after noise addition

[0046] Cluster Set

[0047] Aggregate weight for each class

[0048] The number of clusters C;

[0049] Each client of class c is selected

[0050] Each cluster c is selected by the clustering client

[0051] The knowledge sent by each class c

[0052] The knowledge received by the parameter server

[0053] The knowledge f aggregated by the parameter server t ;

[0054] Knowledge after parameter server selection

[0055] The probability of client selection κ;

[0056] Confidence threshold φ.

[0057] Please see Figure 1 This embodiment provides a method for constructing a heterogeneous federated fine-tuned language model based on differential privacy. The client-local model of the differentially private heterogeneous federated fine-tuned language model (DP-FedLoRA) is obtained through training. The training process specifically includes the following steps:

[0058] Step 1: The parameter server clusters all participating clients according to the resource capabilities of each client and assigns a low-rank adaptive matrix module of corresponding rank to each cluster;

[0059] Step 2: At the beginning of each round of training of the client local model, the server selects a set of participating clients and corresponding clustered clients for each class. Each selected client trains and updates the local model multiple times.

[0060] Step 3: After the selected client completes the local model update, it performs privacy fine-tuning, adds sensitivity-aware noise and regularization optimization to the model update, and sends the privacy-preserving model update to the clustering client;

[0061] Step 4: Perform homogeneous aggregation within the class. The clustering clients of each class receive the privacy model updates of other clients in the same class, including only the trainable parameters of the low-rank adaptive matrix and excluding the parameters of the frozen part. Federated average aggregation is performed to obtain the clustering model.

[0062] Step 5: Perform heterogeneous aggregation between classes. The clustering client of each class calculates on the public dataset and sends the knowledge to the parameter server.

[0063] Step 6: The parameter server uses the correlation between the received knowledge of each class as the aggregation weight, performs weight aggregation, and obtains global knowledge;

[0064] Step 7: The parameter server selects the knowledge whose confidence of each public dataset sample is greater than the threshold from the aggregated knowledge and sends it to each clustering client;

[0065] Step 8: Each clustering client performs a knowledge distillation process on the public dataset for the globally selected knowledge received, updates the clustering model, and sends it to each client in the class.

[0066] Step 9: Each client in each cluster receives the model parameters sent by the clustering client PC and updates its local low-rank adaptive module and local model.

[0067] Step 10: When the preset conditions are met, the training ends and the trained client local model is obtained and applied to the heterogeneous federated privacy fine-tuning system.

[0068] In this embodiment, the specific implementation process of the DP-FedLoRA construction method is as follows:

[0069] Server

[0070] 1: Cluster all clients into C categories based on resource feature vectors.

[0071] 2: Assign adaptive matrix initialization of different sizes to each class according to the resource score of each class;

[0072] 3: Traverse each round t∈{1,…,T} and do the following:

[0073] 4: For each class To traverse, do the following:

[0074] 5: Sampling clients with probability κ and clustering clients And distribute it to the clients of the corresponding class;

[0075] 6: Receive from cluster client knowledge

[0076] 7: Perform weighted knowledge aggregation to obtain global knowledge

[0077] 8: Execute knowledge selection and obtain the selected knowledge And send it to all cluster clients;

[0078] 9: End the loop;

[0079] Client

[0080] 10: Traverse each round t∈{1,…,T} and do the following:

[0081] 11: Receive the client selected in this round and clustering clients

[0082] 12: If the current client is selected

[0083] 13: Execute the privacy fine-tuning of Algorithm 2 to get the model update

[0084] 14: If the current client is not a cluster client

[0085] 15: Sending privacy-preserving model updates To cluster clients

[0086] 16: If the current client is a cluster client

[0087] 17: Perform federated average aggregation on the received model updates to obtain

[0088] 18: Update clustering model and calculate and send model knowledge To the parameter server;

[0089] 19: Receive globally aggregated knowledge

[0090] 20: Perform knowledge distillation and update the model And distribute to clients within the class;

[0091] 21: Other clients receive the global model sent by the clustering client

[0092] 22: Update local model

[0093] 23: End the loop;

[0094] Here we also provide another specific implementation method. In order to make full use of the heterogeneous resource capabilities of the client, clustering is usually used as an effective way to enhance resource utilization and is combined with heterogeneous aggregation between classes to improve the efficiency of training. The present invention designs a resource-aware clustering method such as Figure 2 As shown, clustering is based on available resources rather than related attributes of privacy data. A resource feature vector is used to integrate features such as data volume, computing power, storage capacity, and communication capacity. Clients are classified by fuzzy membership, and clients with similar resource capability levels are grouped into the same class. Uncertainty may arise in the clustering process. When the difference between the highest memberships of a client is less than a threshold value, cluster overlap will occur. In order to avoid this uncertainty, the present invention adopts a load balancing strategy, that is, assigning clients to classes with fewer clients. Then, the resource score of each category is calculated. The class with a higher score represents a class with higher resource capability, and a larger rank adaptive matrix is ​​allocated accordingly.

[0095] The specific implementation process is: the parameter server first performs resource-aware clustering on all clients and defines the resource feature vector χ i =[χ i，1 , χ i，2 , χ i，3 , χ i，4 ], where χ i，1 represents the data volume feature, usually the size of the privacy dataset used for fine-tuning, χ i，2 Represents the computing capability characteristics, usually the number of GPU CUDA cores, χ i，3 Represents the storage capacity characteristics, usually the GPU memory size (GB), χ i，4 Represents the communication capability feature, usually bandwidth (Mbps); for features of different dimensions, first normalize them using Min-Max, then randomly select C cluster centers, and calculate the fuzzy membership of each client i with the cluster center c This reflects the probability that the client belongs to cluster c, d(χ i , v c ) represents the distance between client i and cluster center v c distance, m represents the fuzzy factor; then client i is assigned to the cluster with the highest membership; then the cluster center c is updated for each class, and the above process is repeated until the cluster center no longer changes.

[0096] In addition, knowledge distillation is a commonly used method for heterogeneous aggregation. If we directly perform full heterogeneous knowledge distillation, that is, each client is heterogeneous, direct aggregation will lead to poor model performance due to the low level of knowledge. Figure 3As shown, the approach taken is intra-class aggregation followed by heterogeneous aggregation for inter-class knowledge transfer. By aggregating the model updates for each class to build a clustering model, the quality of knowledge is enhanced to a certain extent. Knowledge is then transferred between classes, and the parameter server uses weight aggregation and knowledge selection to enhance the model performance of the local client.

[0097] Here we also provide another specific implementation method, when the parameter server aggregates, it calculates different weights according to the relevance of the knowledge of each class. The weight is determined by the Pearson correlation coefficient between the knowledge. The higher the correlation, the stronger the consistency of the knowledge. The correlation is calculated as in, From public datasets Sample Computed knowledge, Average knowledge across all classes And the correlation coefficient ρ c Normalization is used as the weight of knowledge aggregation. After that, the parameter server normalizes the correlation. After calculating the weights, perform weight aggregation on the knowledge

[0098] Here we also provide another specific implementation method, which can obtain aggregated knowledge on the basis of ensuring consistency by aggregating knowledge weights. However, the aggregated knowledge may also contain some low-quality content, which will mislead the client's training and lead to poor knowledge distillation effect. The present invention proposes a knowledge selection scheme based on confidence, which selects some knowledge with higher confidence for client training instead of sending all knowledge. The calculation of confidence is in When the confidence φi>φ, the current knowledge i is selected, where φ is the confidence threshold. The parameter server integrates the selected knowledge into fs and sends it to the client for the client's knowledge distillation process.

[0099] In this embodiment, the specific implementation process of the privacy fine-tuning algorithm of client i is as follows:

[0100] Client i

[0101] 1:

[0102] 2: For local round number k = 1, ..., K, traverse and do the following:

[0103] 3: Sampling data sample b t，k ∈D i ;

[0104] 4: Regularization term

[0105] 5: Regularization term optimization loss function

[0106] 6: Calculate the gradient

[0107] 7: If the current local round number k = 1, do the following:

[0108] 8:

[0109] 9: Otherwise, do the following:

[0110] 10:

[0111] 11: Update the model

[0112] 12: Local model update

[0113] 13: Calculate the clipping threshold

[0114] 14: For each locally updated element m=1,...,M, do the following:

[0115] 15:

[0116] 16: Cutting model update

[0117] 17: Adding noise

[0118] 18: End the loop;

[0119] 19: Return privacy-preserving model updates

[0120] Differential privacy is a widely used privacy protection technology that can theoretically provide strong privacy protection capabilities and resist inference attacks. However, adding noise to the low-rank adaptive matrix will cause the model performance to degrade more seriously due to the high sensitivity between parameters. The fundamental reason is that it ignores the difference in sensitivity of different parameters. Parameters with lower sensitivity require less noise to protect privacy, while excessive noise will lead to deviation in the model update direction and performance loss. This paper proposes a sensitivity-aware differential privacy noise distribution scheme, such as Figure 4 As shown, less noise is allocated to parameters with lower sensitivity, and more noise is allocated to parameters with higher sensitivity to protect privacy. Overall, (∈, δ)-differential privacy is still satisfied.

[0121] Another specific implementation is provided here, where the noise of sensitivity perception is determined by first estimating the m-th model of each client i and updating the corresponding clipping threshold s. i，m , and then estimate the sensitivity of the model update And calculate the noise level corresponding to each model update where M is the number of parameter updates, σ * is the overall noise level. Overall, for each client, Without compromising the strength of privacy protection, the noise is redistributed, and adaptive noise is added at the scalar granularity based on the sensitivity of different parameters. This protects privacy while taking into account model performance, especially in low-rank adaptive matrix scenarios.

[0122] This embodiment also provides a device for constructing an efficient heterogeneous federated fine-tuning language model based on differential privacy, including:

[0123] one or more processors;

[0124] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the efficient heterogeneous federated fine-tuning language model construction method based on differential privacy.

[0125] The resource-aware clustering, sensitivity-aware differential privacy, and heterogeneous weighted knowledge aggregation and selection proposed in this invention can also be extended to other heterogeneous algorithms or differential privacy protection algorithms, and have good scalability.

[0126] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A method for constructing a heterogeneous federated fine-tuning language model based on differential privacy, characterized by: The heterogeneous federated fine-tuning language model is obtained by training a local client model; the training process specifically includes the following steps: Step 1: The parameter server clusters all participating N clients into C categories based on the resource capabilities of each client, and assigns a low-rank adaptive matrix LoRA of the corresponding level to each category; Step 2: At the beginning of each round of training of the client local model, the parameter server selects the participating client set P for each class c c and a clustering client p c , each selected client trains and updates the local model multiple times; the selected client participates in this round of training, performs low-rank adaptive matrix LoRA fine-tuning on the local model on its own private dataset, and only updates the local trainable parameters w i , freeze the pre-training weight W0; Step 3: After the selected client completes the local model update, it performs privacy fine-tuning, adds parameter-sensitive differential privacy noise and regularization term optimization to the local model update, and sends the privacy-preserving privacy model update to the clustering client p c ; Step 4: Perform homogeneous aggregation within the class; for each class c, the clustering client p of each class c Received from other clients i∈P in the same class c Privacy model update in Include only LoRA trainable parameters, excluding the parameters of the frozen part, and perform federated average aggregation of FedAvg to obtain the clustering model w c =[A c , B c ]; among them, A c , B c are respectively the aggregated w c The corresponding two low-rank matrices; Step 5: Perform heterogeneous aggregation between classes; the clustering client of each class is in the public dataset Computational knowledge c and sends the knowledge to the parameter server for subsequent heterogeneous LoRA aggregation; Step 6: The parameter server first calculates the aggregation weight ω based on the correlation between the knowledge received for each class. c , then perform weight aggregation to obtain global knowledge F; Step 7: The parameter server selects the knowledge f whose confidence of each public dataset sample is greater than the threshold φ from the aggregated knowledge s , and sent to each clustering client; Step 8: Each cluster client p c , c≤C for the received global selected knowledge f s , perform the knowledge distillation process on the public dataset and update the clustering model w c , and sent to each client P in the class c ; Step 9: The clients in each cluster receive the cluster client p c The clustering model w issued c Parameters, update your local LoRA module w i and the local model W i =W o +B c A c .

2. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 1, characterized in that: In step 1, the parameter server first performs resource-aware clustering on all clients and defines the resource feature vector χ i =[χ i，1 , χ i，2 , χ i，3 , χ i，4 ], where χ i，1 represents the data volume feature, χ i，2 represents the computing capability characteristics, χ i，3 represents the storage capacity characteristic, χ i，4 Represents communication capability characteristics; For features of different dimensions, first normalize them using Min-Max, then randomly select C cluster centers, and calculate the fuzzy membership of each client i with the cluster center c represents the probability that the client belongs to cluster c, d(χ i , v c ) represents the distance between client i and cluster center v c The distance, m is a fuzzy factor greater than 1; Then client i is assigned to the cluster with the highest membership; then the cluster center c is updated for each class, and the above process is repeated until the cluster center no longer changes.

3. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 1, characterized in that: In step 3, after the local fine-tuning is completed, the client updates the local model Δw i Each element of adds sensitivity-aware differential privacy, and each value adds a noise size related to its own sensitivity.

4. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 3 is characterized by: The sensitivity-aware differential privacy is achieved by first estimating the corresponding clipping threshold s for each client i’s m-th local model update. i，m , and then estimate the sensitivity of the model update And calculate the noise level corresponding to each model update where M is the number of parameter updates, σ * is the overall noise level.

5. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 1, characterized in that: In step 6, the parameter server calculates the relevance of the knowledge sent by each cluster client. Here, the Pearson correlation coefficient is used as the weight measurement indicator, and the average knowledge of all clusters is used as a reference. The greater the correlation, the greater the aggregation weight assigned.

6. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 5, characterized in that: The Pearson correlation coefficient in, From public datasets Sample Computed knowledge, is the average clustering knowledge of all samples, Indicates that for the sample The average knowledge of all C clusters; the average knowledge of all classes And the correlation coefficient ρ c Normalization is used as the weight of knowledge aggregation; after that, the parameter server normalizes the correlation coefficient. Calculate the weight ω c After that, perform weight aggregation on the knowledge 7. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to claim 1, characterized in that: In step 7, the parameter server calculates the confidence of each knowledge after aggregation in Representation sample The corresponding knowledge Represents the probability distribution after probability normalization; when the confidence φi>φ, where φ is the confidence threshold.

8. The method for constructing a heterogeneous federated fine-tuning language model based on differential privacy according to any one of claims 1 to 7, characterized in that: In step 8, each clustering client pc receives the selected global knowledge f fed back by the parameter server s , perform the knowledge distillation process, the loss function is in is f s Corresponding to the selected data set, ξ is the adjustment factor, τ is the distillation temperature, and σ is the activation function; Represents a sample, represents the sample, W c are the model parameters of cluster c, is the cross entropy loss function, is the global knowledge of server selection, Is cluster c in the sample Knowledge on D KL () represents the KL divergence loss function.

9. A heterogeneous federated fine-tuning language model construction system based on differential privacy, characterized by: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method for constructing a heterogeneous federated fine-tuning language model based on differential privacy as described in any one of claims 1 to 8.

Images