A method for privacy-preserving fine-tuning and security testing of large models

Abstract

This invention discloses a method for privacy-preserving fine-tuning and security testing of large models, belonging to the field of artificial intelligence technology. The method includes: Step 1: The data provider, in collaboration with an edge coordinator, generates an edge-assisted model based on edge-end federated pre-training and distills privacy data based on multi-spatial data feature matching; Step 2: The data provider generates distilled data with an invisible backdoor; Step 3: The task initiator fine-tunes the target large model in the cloud and performs security testing. This invention aims to protect the privacy of the data provider's private data, reduce the scale of fine-tuning data, and achieve security testing of the large model fine-tuning process.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for privacy-preserving fine-tuning and security testing of large models. Background Technology

[0002] Cloud-based target models trained on public data have demonstrated excellent performance across many domains, but they still struggle to meet the higher demands of clients in certain specialized fields. To further enhance the capabilities of cloud-based target models, a common approach is model fine-tuning. Data providers possess a wealth of usable private data, which can effectively improve the specialized capabilities of cloud-based target models. However, direct use not only increases the risk of privacy breaches for data providers but also consumes significant resources and time. Therefore, efficiently utilizing this data to fine-tune cloud-based target models while protecting private data privacy is a valuable problem, one that has not yet been fully explored.

[0003] Malicious data providers have extensively studied the use of data contamination to launch backdoor attacks. These backdoor attacks are primarily categorized into visible and invisible backdoor attacks. Visible backdoor attacks are difficult to pass manual review due to their obvious nature, while invisible backdoor attacks have garnered significant attention because they can easily evade manual scrutiny. A common method for generating invisible backdoors involves adversarial training to create backdoor generators from clean data. However, the quality of these generators varies, and malicious data providers may employ filtering mechanisms to obtain the highest-quality backdoor generator, posing a greater security threat to the target large-scale model. Summary of the Invention

[0004] Purpose of the invention: In order to address the shortcomings and deficiencies of existing methods, the purpose of this invention is to provide a method for privacy protection fine-tuning and security testing of large models.

[0005] Technical Solution: This invention discloses a method for privacy-preserving fine-tuning and security testing of a large model, comprising the following three stages: the data provider collaborates with an edge coordinator to generate an edge-assisted model for feature extraction based on edge-end federated pre-training and distills privacy data based on multi-spatial data feature matching; the data provider generates distilled data with an invisible backdoor; and the task initiator fine-tunes the target large model in the cloud and performs security testing.

[0006] The data provider DP collaborative edge coordinator ES, based on edge-end federated pre-training to generate feature extraction edge-aid models and based on multi-spatial data feature matching distillation for privacy data, mainly includes the following steps:

[0007] (1a) The edge coordinator ES determines the initial edge auxiliary model F(w0) network depth L based on the data volume set |D|={|D1|,|D2|,...|Dn|} of the data provider DP through the depth selection function Φ.

[0008] (1b) The edge coordinator ES further adjusts the initial edge auxiliary model F(w0) based on the task requirements of the target large model, and then distributes the initial edge auxiliary model F(w0) to n data providers DP. The i-th data provider DP i Based on local data y i Train the model weights independently, and only upload the optimized weights w. i The edge coordinator (ES) aggregates the weights W = {w1, w2, ..., w} of n data providers (DP). n The process involves obtaining a new edge-assisted model and distributing it; after multiple rounds of interactive iterations, a final edge-assisted model F(w) capable of extracting more distribution features is obtained. This process protects the data privacy of the data provider DP while fully utilizing the data from multiple data owners to enable the edge-assisted model to have stronger feature extraction capabilities and improve data distillation results.

[0009] (1c) Data provider DP on local data y i Initial distillation data x is generated after splicing and enhancement. i The initial distillation data x is continuously optimized by using an edge still C based on the final edge-assisted model F(w). i This allows the data to gradually approximate the feature distribution of the original data across multiple spaces, resulting in the final distillation data X. i And upload it to the edge coordinator for aggregation.

[0010] The data provider generates distillation data with an invisible backdoor, which mainly involves the following steps:

[0011] (2a) Simulate a malicious data provider using an adversarial generative network GD (consisting of a generator G and a discriminator D) and local clean data y t Multiple candidate invisible perturbation backdoor generators G*={G*1,G*2,...G* are obtained through training. k These generators inject invisible perturbations into the local distillation data as backdoors.

[0012] (2b) Simulate a malicious data provider DP t Using a backdoor generator selector V, the optimal backdoor generator Gs* is selected for backdoor implantation, generating distilled data with an invisible backdoor, i.e., simulated contaminated data, which is then uploaded to the edge coordinator ES. This simulated contaminated data is more aggressive than the contaminated data generated by the other candidate generators.

[0013] The process of fine-tuning the cloud-based target model and conducting security testing by the task initiator mainly includes the following steps:

[0014] (3a) The task initiator CS loads the pre-trained cloud target large model M and uses efficient parameter fine-tuning methods such as low-rank adaptation (LoRA) to update only the newly added low-rank matrices A and B while keeping the original weight matrix W0 unchanged; the training data comes from the distilled data gathered by the edge coordinator, including clean data and simulated contaminated data.

[0015] (3b) After the task initiator completes the fine-tuning, the evaluation is carried out on the clean test set and the contaminated test set respectively to obtain test results such as the original task performance (ACC) and backdoor attack success rate (ASR). Security analysis is carried out based on the test results.

[0016] As a further preferred embodiment of the present invention, the initial edge-assistance model F(w0) mentioned in step (1a) is a feature learning model, and its main structure is as follows:

[0017] (1),

[0018] (2),

[0019] Where h j Represents the feature output of the j-th layer, ReLU represents the non-linear activation function, IN represents instance normalization to balance the impact of different data providers, Conv represents the convolutional neural network, and w j Let |D| represent the weights of the j-th layer of the neural network, L represent the network depth, and the set |D| = {|D1|,|D2|,...|D...} n |} represents the set of data volumes from n data providers DP, and Φ represents the depth selection function.

[0020] As a further preferred embodiment of the present invention, step (1b) includes the following steps:

[0021] (1b1) The edge coordinator ES sets the output of the initial edge auxiliary model F(w0) according to the task requirements of the target large model, so that its prediction results are adapted to the target task, and distributes the initial edge auxiliary model F(w0) to the data provider DP.

[0022] (1b2) The i-th data provider DP i Using local data y i The optimized weights w are obtained through local training. i The weight set W = {w1, w2, ..., w} is uploaded to the edge coordinator. n}, where n is the number of participating data provider DPs, w i DP is the i-th data provideri Training weights.

[0023] (1b3) The edge coordinator ES collects the weight set W={w1,w2,…,w n The new global weight w* is obtained through aggregation, and the aggregation formula is as follows:

[0024] (3),

[0025] Then, new weights are distributed to the data provider DP, where n is the number of participating data provider DPs;

[0026] (1b4) After multiple weight interactions between the edge coordinator ES and the data provider DP, the trained final edge auxiliary model F(w) is obtained, where w is the final weight parameter, and w is distributed to each data provider DP;

[0027] As a further preferred embodiment of the present invention, the distillation data using the edge still C in step (1c) includes the following steps:

[0028] (1c1) Each data provider DP will transfer local data y i Several images were extracted from each data category and randomly stitched together to obtain the initial distillation data x. i , where x i Let y represent the random initial distillation data from the i-th data provider. i This represents the local data of the i-th data provider.

[0029] (1c2) Each data provider extracts x using the final edge-assisted model F(w). i The features obtained are F(w;x) i ) and y i F(w;y i ), by calculating F(w;x i ) and F(w;y i The mean square error, cosine distance, and complex space characteristic differences of x are obtained. i With y i The total feature distribution matching difference loss is calculated using the following formula:

[0030] (4),

[0031] (5),

[0032] (6),

[0033] (7),

[0034] (8),

[0035] Where α, β, and λ are hyperparameters. 1 represents the mean square error. 2 represents the cosine distance. 3. Differences in the characteristics of complex spaces The loss is the feature distribution matching difference loss, where t is the frequency parameter of the feature transformation, j is an imaginary number, and E is the expected value. and x i and y i The complex space characteristic transformation function, and They are respectively and The conjugate of complex numbers, For the natural index, DP is the i-th data provider. i The edge distiller C minimizes the feature matching loss. Continuously optimize the initial distillation data x i After multiple rounds, the final distillation data X, which concentrates the local data and reveals multiple spatial features, is obtained. i Each data provider will X i Uploaded to the edge coordinator ES, forming a distillation dataset X={X1,X2,…,X…} n}

[0036] As a further preferred embodiment of the present invention, step (2a) includes the following steps:

[0037] (2a1) A malicious data provider initializes the Generative Adversarial Network (GD), where G represents the generator in GD and D represents the discriminator. G uses a multi-layered encoder-decoder (U-Net) structure, which consists of multiple downsampling modules (encoders) and upsampling modules (decoders) of equal number. The forward propagation of the upsampling modules performs channel-dimension skip connections on the feature maps extracted by the downsampling modules at the same level, fusing multi-scale information. The binary classification discriminator D is a simple and effective DCGAN-style discriminator constructed from convolutional neural networks.

[0038] (2a2) Simulate a malicious data provider DP t Clean local data y t The input generator G produces an adversarial backdoor, which is then superimposed on local clean data to obtain contaminated data y. t * Calculate clean data y t and pollution data y t * Mean square error loss 4. Clean up the data yt and pollution data y t * The data is input into a binary classifier D, where the clean data y is... t The label is set to 1, and the contaminated data is y. t * The label is set to 0, and the discriminator loss is calculated based on the output of the binary classifier D. 5. Based on the binary classifier D, analyze the pollution data y t * The output results and 4. Calculate the generator G loss 6. The three loss calculation formulas are as follows:

[0039] (9),

[0040] (10)

[0041] (11),

[0042] Where η and ξ are hyperparameters used to balance mean squared error and adversarial loss, preventing gradient descent from missing the optimal backdoor generator too quickly, and E is the expected value. By minimizing 5 and 6. To optimize and update the generator G and the binary classifier D, after multiple rounds of training, a set G of backdoor generators with different weight perturbations generated in different rounds is obtained. * ={G * 1,G * 2,...G * k}, where k is the number of backdoor generators, and each generator in the set... 4 are all less than the threshold T, which improves the concealment of backdoor implantation.

[0043] As a further preferred embodiment of the present invention, step (2b) includes the following steps:

[0044] (2b1) Simulate a malicious data provider DP t By loading a pre-trained model V as a backdoor generation selector, model V can effectively extract semantic feature information from clean and polluted data, such as OpenAI's open-source multimodal model CLIP.

[0045] (2b2) ​​Simulate a malicious data provider using model V to extract clean data y. t and pollution data y t * The semantic information feature vector V(y) t ) and V(y t* ), calculate V(y t ) and V(y t * )), and the similarity between the eigenvectors is calculated as follows:

[0046] (12),

[0047] Calculate the similarity between the contaminated data and the clean data of different generators in the set G * = {G * 1, G * 2,..G * t. ....G * k} of the perturbation backdoor generators to obtain the similarity set Q = {q1, q2... q t ....q k}, and select the smallest element q s in the similarity set Q, and the corresponding G * in the set G s * of the perturbation backdoor generators is the final perturbation backdoor generator, where s ∈ [0, k]. It can be carried out synchronously with (2a).

[0048] (2b3) The malicious data provider uses the final perturbation backdoor generator G s * to implant an adversarial perturbation backdoor into the distilled data of the malicious data provider, obtain simulated contaminated data, and upload it to the edge coordinator.

[0049] As a further preference of the present invention, step (3a) includes the following steps:

[0050] (3a1) Obtain the pre-trained cloud target large model M, and for a certain trainable weight matrix W0 ∈ R^[d×k] in the target large model M, initialize a pair of low-rank matrices A ∈ R^(r×d) and B ∈ R^(k×r), where the rank r << min(d, k); replace some of the original trainable weight matrices of the target large model with multiple pairs of low-rank matrices A and B to obtain an initial fine-tuning model.

[0051] ​​​(3a3) The task initiator feeds clean test data and test data with backdoors into the fine-tuned target large model for inference, and calculates test results such as the original task performance (ACC) and backdoor attack success rate (ASR) based on the inference results to assess security risks.

[0053] Compared to existing technologies, the beneficial effects of this invention are as follows: This invention provides a method for efficiently utilizing the privacy data of data providers to fine-tune a target large model in the cloud through heterogeneous large model collaboration, thereby improving its generalization and security testing capabilities. This method has the following advantages:

[0054] 1. By collaborating with the data provider through an edge coordinator, the system fully leverages the provider's vast dataset to train an edge-assisted model with superior feature extraction capabilities compared to training solely on local data. The data provider then uses this edge-assisted model to distill their local data, enhancing the distillation process. Data distillation significantly reduces the amount of data while preserving its key features, minimizing resource overhead and time costs for the target large model. Furthermore, the data provider shares only the distilled data, effectively reducing the risk of privacy breaches.

[0055] 2. During the distillation process, the mean square error and cosine distance of the features extracted by the edge-assisted model are added as Euclidean space feature differences as losses to compensate for the singleness of complex space feature differences, thereby improving the effect of data distillation.

[0056] 3. By evaluating the differences between contaminated and clean data in the semantic space through a pre-trained model, the best invisible backdoor generator can be selected, effectively avoiding the misselection of the invisible backdoor generator and resulting in errors in security risk assessment. Attached Figure Description

[0057] Figure 1 This is a system architecture diagram of the present invention;

[0058] Figure 2 Flowchart for generating the edge-assisted model of this invention;

[0059] Figure 3 This is a data distillation flowchart provided by the data provider for this invention.

[0060] Figure 4 This invention provides a flowchart for simulating the injection of a backdoor into a malicious data provider.

[0061] Figure 5 The flowchart for fine-tuning the target large model of this invention is shown. Detailed Implementation

[0062] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0063] For ease of description, the relevant symbols are defined as follows:

[0064] DP represents the data provider, ES represents the edge coordinator, and CS represents the task initiator; F(w0) represents the initial edge-assisted model, and w0 represents the initial weights of model F; the weight set W = {w1, w2, …, w} after training by the data provider. n}, where n is the number of participating data provider DPs, w i For the i-th participating data provider DP i Post-training weights, w* represents the new weights after aggregating the weight set W of the edge coordinator ES; F(w) represents the final trained edge-aided model; y i DP is the i-th data provider i Local data, x i DP is the i-th data provider i Initial distillation data, F(w;x) i ) and F(w;y i ) indicates that x is extracted from F(w). i and y i Features; X = {X1, X2, …, X n} represents the set of distillation data uploaded to the edge coordinator, where X i Let y represent the distillation data provided to the data provider at the i-th position, C represent the edge distiller, G represent the generator of the adversarial network GD, D represent the judge of the adversarial network GD, and V represent the backdoor generation selector. t This indicates clean local data from a malicious data provider, y t * This indicates that a malicious data provider has contaminated the data, V(y) t ) and V(y t * ) indicates that V extracts clean data y t and pollution data y t * The semantic information feature vector, Q = {q1,q2...q k} represents the set of semantic information feature vector similarities between contaminated and clean data with different weights G.

[0065] Figure 1 This is a system framework diagram of the present invention, describing the logic of data exchange and processing between the data provider DP, the edge coordinator ES, and the task initiator CS. The data provider DP and the edge coordinator ES collaboratively train to obtain the final edge-assisted model F(w), where the i-th data provider DP... i Responsible for using local data y iTrain the edge-aided model F(w0) issued by the edge coordinator ES, and then apply the trained weights w i Uploaded to the edge coordinator ES, the edge coordinator ES will upload weights W={w1, w2,…, w n After aggregation, the unified edge-aid model weights are obtained and distributed to the data provider DP. After multiple rounds of interaction, the data provider DP obtains the final edge-aid model F(w). i Local data y is processed by an edge distiller C constructed using an edge-assisted model F(w). i Obtain distillation data x i Malicious data provider DP t Backdoor generator G is used to implant backdoors into distillation data. The distillation data X from data provider DP is transmitted to task initiator CS through an edge coordinator. Task initiator uses the distillation dataset containing backdoor samples provided by the edge coordinator to fine-tune the cloud target model and test the generalization and security of the cloud target model.

[0066] This embodiment discloses a method for privacy-preserving fine-tuning and security testing of a large model, comprising the following three stages: The data provider collaborates with an edge coordinator to generate an edge-assisted model based on edge-end federated pre-training and distills privacy data based on multi-spatial data feature matching; the data provider generates distilled data with an invisible backdoor; the task initiator fine-tunes the target large model in the cloud and performs security testing; such as... Figure 2 As shown, the data provider collaborative edge coordinator generates a feature extraction edge-assisted model based on edge-end federated pre-training, which includes the following steps:

[0067] (1) The edge coordinator (ES) distributes the initial edge auxiliary model F(w0) to the data provider (DP), where F represents the edge auxiliary model and w0 represents the initial weights of model F. F(w0) is the feature learning model, and its main structure is as follows:

[0068] (1),

[0069] (2),

[0070] Where h j Represents the feature output of the j-th layer, ReLU represents the non-linear activation function, IN represents instance normalization to balance the impact of different data providers, Conv represents the convolutional neural network, and w j Let |D| represent the weights of the j-th layer of the neural network, L represent the network depth, and the set |D| = {|D1|,|D2|,...|D...} n |} represents the set of data volumes from n data providers DP, and Φ represents the depth selection function.

[0071] (2) Data provider DP i Using local data y i The optimized weights w are obtained through local training. i The weight set W = {w1, w2, …, w} is uploaded to the edge coordinator. n}, where n is the number of participating data provider DPs, w i DP is the i-th data provider i Training weights.

[0072] (3) The edge coordinator ES collects the weight set W={w1, w2, …, w n The new global weight w* is obtained through aggregation, and the aggregation formula is as follows:

[0073] (3),

[0074] Then, new weights are distributed to the data provider DP, where n is the number of participating data provider DPs;

[0075] (4) After multiple weight interactions between the edge coordinator ES and the data provider DP, the trained final edge auxiliary model F(w) is obtained, where w is the final weight parameter, and w is distributed to each data provider DP.

[0076] like Figure 3 As shown, the process of distilling privacy data based on multiple spatial data features includes the following steps:

[0077] (1) Each data provider DP will transfer local data y i Several images were extracted from each data category and randomly stitched together to obtain the initial distillation data x. i , where x i Let y represent the random initial distillation data from the i-th data provider. i This represents the local data of the i-th data provider.

[0078] (2) Each data provider uses the edge-assisted model F(w) to extract x i The features obtained are F(w;x) i ) and y i F(w;y i ), by calculating F(w;x i ) and F(w;y i The mean square error, cosine distance, and complex space characteristic differences of x are obtained. i With y i The total feature distribution matching difference loss is calculated using the following formula:

[0079] (4),

[0080] (5),

[0081] (6),

[0082] (7),

[0083] (8),

[0084] Where α, β, and λ are hyperparameters. 1 represents the mean square error. 2 represents the cosine distance. 3. Differences in the characteristics of complex spaces The loss is the feature distribution matching difference loss, where t is the frequency parameter of the feature transformation, j is an imaginary number, and E is the expected value. and x i and y i The complex space characteristic transformation function, and They are respectively and The conjugate of complex numbers, The natural index. The i-th data provider DP i The edge distiller C minimizes the feature matching loss. Continuously optimize the initial distillation data x i After multiple rounds, the final distillation data X, which concentrates the local data and reveals multiple spatial features, is obtained. i Each data provider will X i Uploaded to the edge coordinator ES, forming a distillation data set X = {X1, X2, …, X} n}

[0085] like Figure 4 As shown, the data provider generates distillation data with an invisible backdoor by including the following steps:

[0086] (1) The adversarial generative network GD is initialized by a simulated malicious data provider. G represents the generator in the adversarial generative network GD, and D represents the discriminator in the adversarial network GD. G uses a multi-layer encoder-decoder (U-Net) structure. The U-Net consists of multiple downsampling modules (encoders) and upsampling modules (decoders) of the same number. The forward propagation of the upsampling modules will perform channel-dimension skip connections on the feature maps extracted by the downsampling modules at the same level, thus fusing information at multiple scales. The binary classification discriminator D is a simple and effective DCGAN-style discriminator composed of convolutional neural networks.

[0087] (2) Simulate a malicious data provider DPt Clean local data y t The input generator G produces an adversarial backdoor, which is then superimposed on local clean data to obtain contaminated data y. t * Calculate clean data y t and pollution data y t * Mean square error loss 4. Clean up the data y t and pollution data y t * The data is input into a binary classifier D, where the clean data y is... t The label is set to 1, and the contaminated data is y. t * The label is set to 0, and the discriminator loss is calculated based on the output of the binary classifier D. 5. Based on the binary classifier D, analyze the pollution data y t * The output results and 4. Calculate the generator G loss 6. The three loss calculation formulas are as follows:

[0088] (9),

[0089] (10)

[0090] (11),

[0091] Where η and ξ are hyperparameters used to balance mean squared error and adversarial loss, preventing gradient descent from missing the optimal backdoor generator too quickly, and E is the expected value. By minimizing 5 and 6. To optimize and update the generator G and the binary classifier D, after multiple rounds of training, a set G of backdoor generators with different weight perturbations generated in different rounds is obtained. * ={G * 1,G * 2,...G * k}, where k is the number of backdoor generators, and each generator in the set... 4 are all less than the threshold T, which improves the concealment of backdoor implantation.

[0092] (3) Simulate a malicious data provider DP t By loading a pre-trained model V as a backdoor generation selector, model V can effectively extract semantic feature information from clean and polluted data, such as OpenAI's open-source multimodal model CLIP.

[0093] (4) Simulate a malicious data provider using model V to extract clean data y respectively. t and pollution data y t * The semantic information feature vector V(y) t ) and V(y t * ), calculate V(y) t ) and V(y t * The similarity between feature vectors is calculated using the following formula:

[0094] (12),

[0095] Calculate the set G of perturbation backdoor generators respectively. * ={G * 1,G * 2,..G * t. ....G * k The similarity between contaminated and clean data generated by different generators is used to obtain a similarity set Q = {q1,q2,...q}. t ....q k}, select the element q with the smallest similarity in the similarity set Q. s The corresponding set of perturbation backdoor generators G * G in s * For the final perturbation backdoor generator, s∈[0,k].

[0096] (5) Malicious data providers use G s * An anti-disturbance backdoor is implanted into the distillation data of a malicious data provider to obtain simulated contaminated data, which is then uploaded to the edge coordinator.

[0097] like Figure 5 As shown, the steps involved in fine-tuning the large cloud-based target model and conducting security testing by the task initiator are as follows:

[0098] (1) Obtain the pre-trained cloud target large model M. For a certain trainable weight matrix W0∈R^[d×k] in the target large model M, initialize a pair of low-rank matrices A∈R^(r×d) and B ∈R^(k×r), where the rank r << min(d,k); replace some of the original trainable weight matrices of the target large model with multiple pairs of low-rank matrices A and B to obtain the initial fine-tuning model.

[0099] (2) The task initiator CS obtains and organizes the fine-tuning dataset X from the edge coordinator ES, and uses the organized dataset for fine-tuning. During the fine-tuning process, the weight matrix W0 is kept fixed, and only the low-rank matrices A and B are updated; the product of the updated low-rank matrices A and B is added to the original weight matrix W0 to obtain the fine-tuned weight matrix W0'=W0+ΔW=W0+B·A;

[0100] (3) The task initiator will use clean test data and test data with backdoors to perform inference on the fine-tuned target large model, and calculate test results such as the original task performance (ACC) and backdoor attack success rate (ASR) based on the inference results to assess security risks.

[0101] As shown in Table 1, the task initiator effectively improved the original task performance (ACC) by fine-tuning the model using distilled data, and the backdoor attack success rate (ASR) confirms the existence of backdoor attack risk.

[0102] Table 1 - Results of Fine-tuning the Target Model

[0103]

[0104] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for privacy-preserving fine-tuning of large models and security testing, characterized in that, Comprising the following steps: Step one: the data provider cooperates with the edge coordinator to generate a feature extraction edge auxiliary model based on edge-end federation pre-training and distill private data based on multiple spatial data feature matching; Step two: the data provider generates distillation data with invisible backdoor; Step three: the task initiator fine-tunes the cloud target large model and performs security testing; The step one comprises the following steps: (1a) The edge coordinator ES determines the data volume set |D|={|D1|,|D2|,...|D...} based on the data volume set of the data provider DP. n |}, Determine the initial edge auxiliary model F(w0) network depth L through the depth selection function Φ; (1b) The edge coordinator ES further adjusts the initial edge auxiliary model F(w0) according to the task requirements of the target large model, and then distributes the initial edge auxiliary model F(w0) to n data providers DP, the i-th data provider DP i Based on local data y i Train the model weight independently, and only upload the optimized weight w i ; The edge coordinator ES aggregates the weights W={w1, w2, …, w n} of n data providers DP to obtain a new edge auxiliary model and issue it; After multiple rounds of interaction iteration, the final edge auxiliary model F(w) capable of extracting more distribution characteristics is obtained; (1c) Data provider DP provides local data y i After splicing and enhancement, the initial distillation data x is generated i And through the edge distiller C based on the final edge auxiliary model F(w), the initial distillation data x is continuously optimized i So that it gradually approximates the original data feature distribution in multiple spaces, and the final distillation data X is obtained i And uploaded to the edge coordinator for summary; The step two comprises the following steps: (2a) Simulate a malicious data provider using a generative adversarial network GD and local clean data y t , train a plurality of candidate invisible perturbation backdoor generators G * ={G * 1,G * 2,...G * k}; the candidate invisible perturbation backdoor generator injects invisible perturbation into the local distillation data as a backdoor; (2b) Simulate malicious data provider DP t With backdoor generation selector V, select the best backdoor generator G s * For backdoor implantation, generate distilled data with invisible backdoor, upload it to edge coordinator ES; The step three comprises the following steps: (3a) The task initiator CS loads the pre-trained cloud target large model M, and adopts the parameter fine-tuning method to update only the newly added low-rank matrices A and B while keeping the original weight matrix W0 unchanged; the training data is derived from the distillation data collected by the edge coordinator, including clean data and simulated contaminated data; (3b) After the task initiator completes the fine-tuning, evaluate on the clean test set and the contaminated test set respectively to obtain the test results, and perform security analysis according to the test results.

2. The method of claim 1, wherein, The initial edge auxiliary model F(w0) in step (1a) is a feature learning model, and the structure is as follows: （1）， （2）， where h j represents the feature output of the jth layer, ReLU represents a nonlinear activation function, IN represents instance normalization, Conv represents a convolutional neural network, w j represents the jth layer neural network weight, L represents the network depth, the set |D|={|D1|,|D2|,...|D n |} represents a data volume set of n data providers DP, and Φ represents a depth selection function.

3. The method of claim 2, wherein, Step (1b) comprises the following steps: (1b1) The edge coordinator ES sets the output of the initial edge auxiliary model F(w0) according to the task demand of the target large model, so that the prediction result is adapted to the target task, and distributes the initial edge auxiliary model F(w0) to the data provider DP; (1b2) the i-th data provider DP i with local data y i i n i i train the weights;​​​​ (1b3) The edge coordinator ES collects the weight set W={w1,w2,…,w n The new global weight w* is obtained through aggregation, and the aggregation formula is as follows: （3）， Then distribute the new weight to the data provider DP, where n is the number of participating data providers DP; (1b4) After multiple weight interactions between the edge coordinator ES and the data provider DP, the final trained edge auxiliary model F(w) is obtained, where w is the final weight parameter, and w is distributed to each data provider DP.

4. The method of claim 3, wherein, The step (1c) using edge distiller C to distill data comprises the following steps: (1c1) each data provider DP extracts local data y i A number of pictures are extracted from each type of data to obtain initial distillation data x i where x i represents the random initial distillation data of the i-th data provider, y i represents the local data of the i-th data provider; (1c2) Each data provider extracts x using the final edge-assisted model F(w). i The features obtained are F(w;x) i ) and y i F(w;y i ), by calculating F(w;x i ) and F(w;y i The mean square error, cosine distance, and complex space characteristic differences of x are obtained. i With y i The total feature distribution matching difference loss is calculated using the following formula: （4）， （5）， （6）， （7）， （8）， where a, b and l are hyperparameters, 1 is the mean square error, 2 is the cosine distance, 3 is the complex space feature difference, is the feature distribution matching difference loss, t is the frequency parameter of feature transformation, j is the imaginary number, and E is the mathematical expectation, and are the complex space feature transformation functions of x i and y i respectively, and are the conjugate complex numbers of and respectively, is the natural exponential, and the edge distiller C of the ith data provider DP i optimizes the initial distilled data x i by minimizing the feature matching loss After multiple rounds, the final distilled data X i that concentrates the multi-space features of the local data is obtained, and each data provider uploads X i to the edge coordinator ES to form a distilled data set X = {X1, X2, …, X n}.

5. The method of claim 4, wherein, Step (2a) comprises the following steps: (2a1) Simulate the initialization of the adversarial generation network GD, G represents the generator in the adversarial generation network GD, D represents the discriminator in the adversarial network GD, G uses a multi-layer U-Net structure, and the U-Net structure is composed of a plurality of down-sampling modules and up-sampling modules with the same number, wherein the forward propagation of the up-sampling module connects the feature maps extracted by the down-sampling module at the same level in the channel dimension, fuses multi-scale information, and the binary classification discriminator D is composed of a DCGAN type discriminator composed of a convolutional neural network; (2a2) simulate malicious data provider DP t The local clean data y t The input generator G gets the adversarial perturbation backdoor, and then superimposes the adversarial perturbation backdoor with the local clean data to get the contaminated data y t * , calculate the mean square error loss of clean data y t and contaminated data y t * 4, input clean data y t and contaminated data y t * to the binary classification discriminator D, wherein the set label of clean data y t is 1, and the set label of contaminated data y t * is 0, and the discriminator loss is calculated according to the output result of the binary classification discriminator D 5, according to the output result of the binary classification discriminator D on the contaminated data y t * and 4, calculate the generator G loss 6, the three loss calculation formulas are as follows:​ （9）， （10）， （11）， where η and ξ are hyperparameters to balance the mean square error and adversarial loss, E is the mathematical expectation, and the update generator G and binary classification discriminator D are optimized by minimizing 5 and 6, and a plurality of different weight perturbation backdoor generator sets G * = {G * 1, G * 2,...G * k} generated by different rounds are obtained after multiple rounds of training, k is the number of backdoor generators, and the 4 of each generator in the set is less than the threshold T.

6. The method of claim 5, wherein: Step (2b) comprises the following steps: (2b1) simulate a malicious data provider DP t load a pre-trained model V as a backdoor generation selector; (2b2) Extracting clean data y t and polluted data y t using model V respectively by malicious data provider * utilization model V t ) and V(y t * ), and calculating the similarity between the feature vectors V(y t ) and V(y t * ). The calculation formula is as follows: （12）， Calculate the similarity of the different generators of the perturbed backdoor generator set G * ={G * 1,G * 2,..G * t. ....G * k} with clean data and the similarity set Q={q1,q2...q t ....q k} is obtained, and the smallest element q s in the similarity set Q is selected. The corresponding G * in the perturbed backdoor generator set G s * is the final perturbed backdoor generator, s∈[0,k]. (2b3) Malicious data provider utilizes the final perturbed backdoor generator G s * Distilled data implantation of adversarial perturbation backdoor to malicious data provider, get simulated contaminated data, upload to edge coordinator.

7. The method of claim 6, wherein, Step (3a) The data used by the task initiator CS large model fine-tuning is the collaborative distillation data X={X1, X2, …, X n}, that is: （13）， denotes a cloud target large model, denotes a large model parameter, denotes a task loss function.

Images