Privacy protection type data joint modeling method based on federal learning

Abstract

The invention relates to the technical field of data protection, and discloses a privacy protection type data joint modeling method based on federated learning, which comprises the following steps: acquiring local data to perform meta-feature extraction, calculating key statistics to characterize data characteristics, collecting meta-features, grouping the meta-features into similar feature clusters through spectral clusters, and carrying out feature clustering on the similar feature clusters; dynamically allocating and calculating resource weights according to the similar characteristic cluster scale and the equipment computing power; distributing a basic privacy budget according to the client type, calculating a local model accuracy rate and an intra-cluster level difference, dynamically adjusting the privacy budget, adding adaptive Gaussian noise based on the privacy budget, and adjusting gradient sensitivity of gradient calculation; verifying gradient compliance through zero knowledge, carrying out safe aggregation on gradients passing verification, optimizing a meta-model through a knowledge distillation loss function, and generating confrontation sample analysis to obtain a leakage risk value to identify knowledge leakage risks; sensitive neurons in the neuron sensitivity positioning element model are analyzed and calculated, directional noise is injected, and initial parameters are adjusted for initialization training.

Description

Technical Field

[0001] The present invention relates to the field of data protection technology, and in particular to a privacy-preserving data joint modeling method based on federated learning. Background Art

[0002] Privacy-preserving data federation modeling is a core technology system for addressing the tightening of data privacy regulations and the need for cross-institutional data collaboration. Its development relies on the deep integration of cryptography, distributed computing, and machine learning. In existing technologies, data is retained on local devices, model gradients or parameters are shared, and cryptographic techniques are used to prevent the server from parsing the gradient information of a single client. Noise is added to local gradients or aggregated results to provide data privacy guarantees. Clients add noise before sending gradients, but the magnitude of the noise affects model convergence. Federated transfer learning is used for joint training across heterogeneous data domains to reduce the need for data alignment. The fundamental contradiction between privacy protection and model effectiveness lies in the fact that noise addition leads to decreased accuracy, rigid privacy budget allocation, a fixed ε value that cannot adapt to the dynamic training process, and excessive noise in early rounds that inhibits model convergence. Attackers use shared gradients to infer the original data. Existing defenses are ineffective against small batches of data. Model updates are used to determine whether specific samples participate in training. Differential privacy noise must reach a destructive level to be defended. Malicious clients can simultaneously steal data and implant model backdoors by constructing special gradients. Frequent joining / exiting causes the security aggregation protocol to be interrupted, requiring key renegotiation. Existing frameworks rely on periodic aggregation and cannot support online learning scenarios. Gradient-based anomaly detection is vulnerable to adaptive attacks. In view of this, it is necessary to provide a privacy-preserving data joint modeling method based on federated learning. Summary of the Invention

[0003] The purpose of the present invention is to provide a privacy-preserving data joint modeling method based on federated learning. To solve the above-mentioned problems in the prior art, the present invention is implemented through the following technical solutions: An embodiment of the present invention provides a privacy-preserving data joint modeling method based on federated learning, which specifically includes the following steps: Step 1: Obtain local data for meta-feature extraction, calculate key statistics to characterize data characteristics, collect meta-features and group them into clusters with similar characteristics through spectral clustering, and dynamically allocate computing resource weights based on the size of the clusters with similar characteristics and the computing power of the device; Step 2: Allocate a basic privacy budget based on the client type, calculate the local model accuracy and the difference between the local model accuracy and the intra-cluster accuracy, dynamically adjust the privacy budget, add adaptive Gaussian noise based on the privacy budget, and adjust the gradient sensitivity of the gradient calculation; Step 3: Verify gradient compliance through zero-knowledge validation and securely aggregate verified gradients. Optimize the meta-model using the knowledge distillation loss function, generate adversarial examples for analysis, and obtain leakage risk values ​​to identify knowledge leakage risks. Step 4: Analyze and calculate the sensitivity of neurons to locate sensitive neurons in the meta-model, inject directional noise, and adjust the initial parameters for initialization training.

[0004] Furthermore, the method for obtaining the similar characteristic cluster is: Extracting four-dimensional feature vectors through the client

[0005] Collect the four-dimensional feature vectors of all clients ,in, is the total number of four-dimensional meta-feature vectors, calculates the Euclidean distance between meta-features, and constructs a similarity matrix

[0006] in, is the bandwidth parameter, which controls the similarity decay speed. and are all four-dimensional element eigenvectors, and is the four-dimensional element feature vector index; Based on the obtained similarity matrix, the degree matrix D is constructed, where the diagonal elements , calculate the Laplace matrix L: in, is the similarity matrix; Solve and obtain the four-dimensional element eigenvectors corresponding to the first K minimum eigenvalues ​​of the Laplace matrix L, map the client to a low-dimensional space, and perform K-means clustering to generate K clusters with similar characteristics , For the clusters of similar features; Furthermore, the method for dynamically allocating computing resource weights is: Get the kth similar feature cluster The number of clients in the k-th similarity cluster is obtained Cluster size ; Get the device's CPU core count, memory size, and network bandwidth, and multiply them by de-dimensionalizing them to get the device's computing capability. ; Cluster size based on similarity characteristics and device computing capability value Allocate resource weights :

[0007] in, Calculate capability values ​​for the average devices within a cluster of similar characteristics, is the total number of similar feature clusters, Indexing clusters of similar features; Furthermore, the method for dynamically adjusting the privacy budget is: The client dynamically adjusts the privacy budget based on the difference between the local model accuracy and the average accuracy within the cluster, using the formula:

[0008] Get the dynamically adjusted privacy budget ; in, As a basic privacy budget, is the local model accuracy, is the average accuracy within the cluster, is the adjustment amplitude factor, is the sensitivity factor; Furthermore, the method for adjusting the gradient sensitivity of the gradient calculation is: Based on the obtained local cross entropy loss, adaptive Gaussian noise is added to the local cross entropy loss gradient; Controlling the gradient through gradient clipping The norm upper limit is the gradient clipping threshold ,Right now , so the gradient sensitivity , ensuring that the noise scale matches the gradient sensitivity, through the formula:

[0009] Get the adaptive cross entropy loss gradient with adaptive noise protection ; in, is the local cross entropy loss of client i with respect to the model parameters The original gradient of is the failure probability, is the identity matrix, is Gaussian noise, To adjust the privacy budget; Furthermore, the method for verifying gradient compliance is: Client generates zero-knowledge proof , is the adaptive cross entropy loss gradient, For the degree matrix, prove the gradient legality and noise compliance; Generate zero-knowledge proof by the client to verify whether the adaptive cross entropy loss gradient is determined by local data Correct calculation; verify that the added noise meets distribution, and:

[0010] in, is the gradient clipping threshold, is the failure probability, The privacy budget is adjusted dynamically; Furthermore, the method for performing security aggregation is: If the gradient legitimacy and noise compliance are verified, the verified gradients are securely aggregated using the formula:

[0011] Calculate the safety aggregation gradient ; in, is the adaptive cross entropy loss gradient, is the total number of gradients that pass the verification, To verify the gradient index, It is an indicator function, which is 1 if the verification is successful, otherwise it is 0; Furthermore, the method for optimizing the meta-model is: After security aggregation is completed, a meta-model is constructed and optimized using the knowledge distillation loss function. Building a lightweight neural network meta-model , input original features and client meta features , output the prediction result of integrating global knowledge, through the knowledge distillation loss function formula:

[0012] Perform knowledge distillation to obtain an optimized meta-model; in, is the total number of client meta-features, is the client meta-feature index, is the KL divergence, is the current global model, is the global model parameter, are the learnable parameters of the meta-model; Furthermore, the method for identifying knowledge leakage risk is: Generate adversarial samples through the fast gradient sign method FGSM, through the formula:

[0013] Get adversarial examples ; in, is the disturbance intensity, is the loss function, is the loss function The gradient of the original sample x is: is the true label corresponding to the original sample x, is a symbolic function, is the global model parameter, is the current global model; Based on the adversarial samples , through the formula:

[0014] The leakage risk value R is calculated. A high leakage risk value R indicates that the model is more sensitive to adversarial samples and there is a risk of knowledge leakage. in, is the total number of samples, is the sample index, is the confidence of the model in the prediction results, For adversarial samples, For the original sample, For the A global model of adversarial examples, For the A global model of adversarial examples, is the global model parameter; Identify knowledge leakage risks based on leakage risk values; Furthermore, the method for performing initialization training is: By formula Get the noise intensity ; in, is the neuronal sensitivity, is the neuron index, is the basic noise standard deviation, To adjust the parameters, is the maximum value of all neuron sensitivities; In the metamodel parameters Directional noise is injected into Calculate the adjustment parameters after injecting directional noise into the initial parameters ,in, are the metamodel parameters of neuron j, is directional noise; Noise was injected into the neurons in the high-sensitivity group, while the others remained unchanged; Obtain client meta-features and historical training performance to dynamically generate personalized initial parameters; Generate a function with initial parameters:

[0015] Get the initial parameters ; in, is the neural network meta-model, is the four-dimensional element feature vector of client i, is the adjustment parameter after injecting directional noise, is the historical training gradient accumulation, is the learning rate parameter; Initialize training based on initial parameters.

[0016] The second aspect is a privacy-preserving data joint modeling system based on federated learning, which specifically includes the following modules: Resource management module: This module obtains local data for meta-feature extraction, calculates key statistics to characterize data characteristics, groups collected meta-features into clusters with similar characteristics through spectral clustering, and dynamically allocates computing resource weights based on the size of similar characteristic clusters and the computing power of the device; Privacy protection module: Allocates a basic privacy budget based on client type, calculates the local model accuracy and the difference between the local model accuracy and the intra-cluster accuracy, dynamically adjusts the privacy budget, adds adaptive Gaussian noise based on the privacy budget, and adjusts the gradient sensitivity of gradient calculation; Training protection module: Verifies gradient compliance through zero-knowledge and securely aggregates verified gradients. It optimizes the meta-model through the knowledge distillation loss function, generates adversarial sample analysis, obtains leakage risk values, and identifies knowledge leakage risks. Model optimization module: Analyzes and calculates neuron sensitivity to locate sensitive neurons in the meta-model, injects directional noise, and adjusts initial parameters for initialization training.

[0017] Beneficial effects of the present invention: 1. By extracting four-dimensional meta-feature vectors to quantify data characteristics, spectral clustering is used to group clients into clusters with similar characteristics. Computing resource weights are dynamically allocated based on cluster size and device computing power. This eliminates the need to transmit raw data and characterizes data distribution characteristics solely through meta-features, reducing data transmission volume. Spectral clustering, combined with Laplace matrix dimensionality reduction, improves clustering accuracy, avoids sensitivity to initial values, and aligns privacy protection strength with data sensitivity and model performance. Gradient clipping is combined with adaptive Gaussian noise to match noise scale with gradient sensitivity, achieving refined adjustments between privacy protection and model efficiency. 2. Use zero-knowledge proof to verify the legitimacy of gradients and compliance with noise to ensure that the aggregation process is not maliciously tampered with; build a meta-model and integrate global knowledge through knowledge distillation. At the same time, use adversarial sample generation and leakage risk value calculation to dynamically identify knowledge leakage risks and adjust privacy policies. By calculating the sensitivity of neurons to input perturbations, directional noise is injected into highly sensitive neurons, while medium and low-sensitivity neurons remain unchanged; combine client meta-features and historical training performance to dynamically generate personalized initial parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flowchart of the steps of a privacy-preserving data joint modeling method based on federated learning provided in Example 1 of the present invention; Figure 2 This is a structural diagram of a privacy-preserving data joint modeling system based on federated learning provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0021] Example 1 like Figure 1 As shown, an embodiment of the present invention provides a privacy-preserving data joint modeling method based on federated learning, which specifically includes the following steps: Step 1: Obtain local data for meta-feature extraction, calculate key statistics to characterize data characteristics, collect meta-features and group them into clusters with similar characteristics through spectral clustering, and dynamically allocate computing resource weights based on the size of the clusters with similar characteristics and the computing power of the device; In a specific embodiment, local data is collected by the client ,in, is the data feature vector, is the data label, is the local sample size, is the data index, is the sample index; Extracting four-dimensional feature vectors through the client , is the real number set of the four-dimensional element eigenvector, and the quantized data characteristics are obtained as the four-dimensional element eigenvector:

[0022] in, is the label mean, reflecting the bias of label distribution; is the feature standard deviation, which measures feature diversity; is the feature skewness, which detects the asymmetry of the feature distribution; is label entropy, which evaluates label uncertainty; For example, local data is obtained from client A to extract the four-dimensional element feature vector , which means that the mean of the data label distribution is 0.7, the standard deviation of the label feature is 0.3, indicating that the data variance is small, the data feature skewness is 0.9, which indicates that the data feature distribution is left-skewed, and the label entropy is 0.88, which indicates that the data label distribution is uniform; obtain the local data from client B to extract the four-dimensional element feature vector , which means that the mean of the data label distribution is 0.2, the standard deviation of the label feature is 0.5, indicating that the data variance is large, the data feature skewness is 1.2, which indicates that the data feature distribution is right-skewed, and the label entropy is 0.32, which indicates that the data label distribution is uneven; Collect the four-dimensional feature vectors of all clients ,in, is the total number of four-dimensional meta-feature vectors, calculates the Euclidean distance between meta-features, and constructs a similarity matrix

[0023] in, is the bandwidth parameter, which controls the similarity decay speed. and are all four-dimensional element eigenvectors, and is the four-dimensional element feature vector index; Based on the obtained similarity matrix, the degree matrix D is constructed, where the diagonal elements , calculate the Laplace matrix L: ,in, is the similarity matrix; Solve and obtain the four-dimensional element eigenvectors corresponding to the first K minimum eigenvalues ​​of the Laplace matrix L, map the client to a low-dimensional space, and perform K-means clustering to generate K clusters with similar characteristics , For the clusters of similar features; Get the kth similar feature cluster The number of clients in the k-th similarity cluster is obtained Cluster size ; Get the device's CPU core count, memory size, and network bandwidth, and multiply them by de-dimensionalizing them to get the device's computing capability. ; Cluster size based on similarity characteristics and device computing capability value Allocate resource weights :

[0024] in, Calculate capability values ​​for the average devices within a cluster of similar characteristics, Index clusters with similar features. Clusters with high computing power and similar features can obtain more computing resources, balancing training efficiency and data contribution. Step 2: Allocate a basic privacy budget based on the client type, calculate the local model accuracy and the difference between the local model accuracy and the intra-cluster accuracy, dynamically adjust the privacy budget, add adaptive Gaussian noise based on the privacy budget, and adjust the gradient sensitivity of the gradient calculation; Identify sensitive data through predefined regular expressions, dictionary libraries, or grammar rules; Mark the clusters of similar features where clients with sensitive data are located as sensitive clusters , mark the similar feature clusters where clients without sensitive data are located as normal clusters ; The data in the sensitive cluster is highly sensitive and has high privacy requirements, so a strict basic privacy budget is allocated. The data in the common cluster is less sensitive and has low privacy requirements, so a loose basic privacy budget is allocated. Specifically, by allocating different basic privacy budgets :

[0025] Perform differentiated privacy protection; Split the local data of client i into a training set for training the model and a validation set that is not involved in training and is only used to evaluate the model; Train the deep learning model based on the training set to obtain the local model of client i; Based on the local model of client i, make predictions on the validation set data, calculate the ratio of the number of correctly predicted samples to the total number of samples in the validation set, and get the accuracy of the local model ; After mapping client i to a low-dimensional space, perform K-means clustering to determine the cluster where client i belongs, and clearly determine whether the client belongs to a sensitive cluster or a common cluster; Get the local model accuracy of each client in the similar feature cluster where client i is located; The average accuracy of the cluster is obtained by summing the local model accuracy of all clients in the cluster with similar characteristics where client i is located and dividing it by the total number of clients in the cluster with similar characteristics where client i is located. ; It should be noted that the purpose of splitting the validation set is to avoid inflated accuracy due to overfitting of the training set and to ensure that the calculated local model accuracy truly reflects the model's predictive ability. If the local model accuracy of client j is much higher than the average accuracy within the cluster, it means that the data quality / model effect of client j is better. By dynamically adjusting the privacy budget, the privacy budget of client j can be reduced to make the model more efficient. Otherwise, the privacy budget can be increased to strengthen privacy protection. The client dynamically adjusts the privacy budget based on the difference between the local model accuracy and the average accuracy within the cluster, using the formula:

[0026] Get the dynamically adjusted privacy budget ; in, As a basic privacy budget, is the local model accuracy, is the average accuracy within the cluster, To adjust the amplitude factor, control the budget change range and avoid over-adjustment, It is a sensitivity factor that amplifies the impact of accuracy differences and quickly responds to changes in data quality; Get the client's prediction output based on the local model and the true label The difference between , calculate the cross entropy loss , through the formula:

[0027] Calculate the local cross entropy loss ; in, is the total number of local samples of client i, is the jth sample, are model parameters, For client index, is the sample index; Based on the obtained local cross entropy loss, adaptive Gaussian noise is added to the local cross entropy loss gradient; Controlling the gradient through gradient clipping The norm upper limit is the gradient clipping threshold ,Right now , so the gradient sensitivity , ensuring that the noise scale matches the gradient sensitivity, through the formula:

[0028] Get the adaptive cross entropy loss gradient with adaptive noise protection ; in, is the local cross entropy loss of client i with respect to the model parameters The original gradient of is the failure probability, is the identity matrix, is Gaussian noise, To adjust the privacy budget; It should be noted that the noise variance is proportional to the square of the adjusted privacy budget Inversely proportional, the smaller the privacy budget, the greater the noise, the stronger the privacy protection, and the noise variance is The more sensitive the gradient, the lower the failure probability, and the greater the noise; Step 3: Verify gradient compliance through zero-knowledge validation and securely aggregate verified gradients. Optimize the meta-model using the knowledge distillation loss function, generate adversarial examples for analysis, and obtain leakage risk values ​​to identify knowledge leakage risks. Client generates zero-knowledge proof , For the degree matrix, prove the gradient legality and noise compliance; Specifically, the client generates a zero-knowledge proof to verify whether the adaptive cross entropy loss gradient is generated by the local data. Correct calculation; verify that the added noise meets distribution, and:

[0029] in, is the gradient clipping threshold, is the failure probability, The privacy budget is adjusted dynamically; If the gradient legitimacy and noise compliance are verified, the verified gradients are securely aggregated using the formula:

[0030] Calculate the safety aggregation gradient ; in, is the total number of gradients that pass the verification, To verify the gradient index, It is an indicator function, which is 1 if the verification is successful, otherwise it is 0; After security aggregation is completed, a meta-model is constructed and optimized using the knowledge distillation loss function. Specifically, build a lightweight neural network meta-model , input original features and client meta features , output the prediction result of integrating global knowledge, through the knowledge distillation loss function formula:

[0031] Perform knowledge distillation to obtain an optimized meta-model; in, is the total number of client meta-features, is the client meta-feature index, is the KL divergence, which measures the difference between the two models. is the current global model, is the global model parameter, are the learnable parameters of the meta-model; Generate adversarial samples through the fast gradient sign method FGSM, through the formula:

[0032] Get adversarial examples ; in, is the disturbance intensity, is the loss function, is the loss function The gradient of the original sample x is: is the true label corresponding to the original sample x, is a symbolic function, is the global model parameter, is the current global model; Based on the adversarial samples , through the formula:

[0033] The leakage risk value R is calculated. A high leakage risk value R indicates that the model is more sensitive to adversarial samples and there is a risk of knowledge leakage. in, is the total number of samples, is the sample index, is the confidence of the model in the prediction results, For adversarial samples, For the original sample, For the A global model of adversarial examples, For the A global model of adversarial examples, is the global model parameter; Identify knowledge leakage risks based on leakage risk values; Specifically, if R exceeds the leakage risk threshold, the privacy budget is increased and adversarial training is added to the loss function. Retrain the meta-model to enhance robustness; Step 4: Analyze and calculate the sensitivity of neurons to locate the sensitive neurons in the meta-model, inject directional noise, and adjust the initial parameters for initialization training; Calculate the sensitivity of neuron j to input perturbations and identify the neuron in the meta-model that is most sensitive to private data using the formula:

[0034] Calculate the sensitivity of neuron j ; in, is the total number of neurons, is the neuron index, is the jth neuron pair sample in the meta-model The output, For the sample, is the metamodel parameter, Output of the neuron About input samples The gradient, for Norm calculation; Sensitivity of all neurons Sort in descending order; K-means clustering was used to classify the top 20% of neurons in descending order as highly sensitive neurons; Arrange the neurons in descending order and classify the middle 50% as medium-sensitive neurons; After sorting the neurons in descending order, 30% are classified as low-sensitivity neurons; Noise is injected into highly sensitive neurons, while noise is not injected into medium and low-sensitivity neurons, thus achieving a balance between privacy protection and model performance. By formula Get the noise intensity ; in, is the basic noise standard deviation, To adjust the parameters and enhance the noise of highly sensitive neurons, is the maximum value of all neuron sensitivities; In the metamodel parameters Directional noise is injected into Calculate the adjustment parameters after injecting directional noise into the initial parameters ,in, are the metamodel parameters of neuron j, is directional noise; Noise was injected into the neurons in the high-sensitivity group, while the others remained unchanged; Obtain client meta-features and historical training performance to dynamically generate personalized initial parameters; Generate a function with initial parameters:

[0035] Get the initial parameters ; in, is the neural network meta-model, is the four-dimensional element feature vector of client i, is the adjustment parameter after injecting directional noise, is the historical training gradient accumulation, is the learning rate parameter; Initialize training based on initial parameters.

[0036] like Figure 2 As shown, the embodiment of the present invention provides, specifically including the following modules: Resource management module: This module obtains local data for meta-feature extraction, calculates key statistics to characterize data characteristics, groups collected meta-features into clusters with similar characteristics through spectral clustering, and dynamically allocates computing resource weights based on the size of similar characteristic clusters and the computing power of the device; Privacy protection module: Allocates a basic privacy budget based on client type, calculates the local model accuracy and the difference between the local model accuracy and the intra-cluster accuracy, dynamically adjusts the privacy budget, adds adaptive Gaussian noise based on the privacy budget, and adjusts the gradient sensitivity of gradient calculation; Training protection module: Verifies gradient compliance through zero-knowledge and securely aggregates verified gradients. It optimizes the meta-model through the knowledge distillation loss function, generates adversarial sample analysis, obtains leakage risk values, and identifies knowledge leakage risks. Model optimization module: Analyzes and calculates neuron sensitivity to locate sensitive neurons in the meta-model, injects directional noise, and adjusts initial parameters for initialization training.

[0037] An embodiment of the present invention is described in detail above, but the content described is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention; the above formulas are all dimensionless and numerical calculations, and the formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field based on actual conditions and historical experience, and can be adjusted according to actual conditions; the above description is only a preferred embodiment of the present invention and is not used to limit the present invention. All equal changes and improvements made according to the scope of application of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A privacy-preserving data joint modeling method based on federated learning, characterized by: The following steps are involved: Obtain local data for meta-feature extraction, calculate key statistics to characterize data characteristics, collect meta-features and group them into clusters with similar characteristics through spectral clustering, and dynamically allocate computing resource weights based on the size of similar characteristic clusters and device computing power; Allocate a basic privacy budget based on client type, calculate the difference between local model accuracy and intra-cluster accuracy, dynamically adjust the privacy budget, add adaptive Gaussian noise based on the privacy budget, and adjust the gradient sensitivity of gradient calculation; Gradient compliance is verified through zero-knowledge verification, and verified gradients are securely aggregated. The meta-model is optimized through the knowledge distillation loss function, and adversarial sample analysis is used to obtain leakage risk values ​​to identify knowledge leakage risks. Analyze and calculate the sensitivity of neurons to locate the sensitive neurons in the meta-model, inject directional noise, and adjust the initial parameters for initialization training.

2. A privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for obtaining the similar characteristic cluster is: Extracting four-dimensional feature vectors through the client ; Collect the four-dimensional feature vectors of all clients ,in, is the total number of four-dimensional meta-feature vectors, calculates the Euclidean distance between meta-features, and constructs a similarity matrix ; in, is the bandwidth parameter, which controls the similarity decay speed. and are all four-dimensional element eigenvectors, and is the four-dimensional element feature vector index; Based on the obtained similarity matrix, the degree matrix D is constructed, where the diagonal elements , calculate the Laplace matrix L: ,in, is the similarity matrix; Solve and obtain the four-dimensional element eigenvectors corresponding to the first K minimum eigenvalues ​​of the Laplace matrix L, map the client to a low-dimensional space, and perform K-means clustering to generate K clusters with similar characteristics , For the clusters of similar features.

3. A privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for dynamically allocating computing resource weights is: Get the kth similar feature cluster The number of clients in the k-th similarity cluster is obtained Cluster size ; Get the device's CPU core count, memory size, and network bandwidth, and multiply them by de-dimensionalizing them to get the device's computing capability. ; Cluster size based on similarity characteristics and device computing capability value Allocate resource weights : ； in, Calculate capability values ​​for the average devices within a cluster of similar characteristics, is the total number of similar feature clusters, Index for clusters of similar features.

4. A privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for dynamically adjusting the privacy budget is: The client dynamically adjusts the privacy budget based on the difference between the local model accuracy and the average accuracy within the cluster, using the formula: ； Get the dynamically adjusted privacy budget ; in, As a basic privacy budget, is the local model accuracy, is the average accuracy within the cluster, is the adjustment amplitude factor, is the sensitivity factor.

5. The privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for adjusting the gradient sensitivity of the gradient calculation is: Based on the obtained local cross entropy loss, adaptive Gaussian noise is added to the local cross entropy loss gradient; Controlling the gradient through gradient clipping The norm upper limit is the gradient clipping threshold ,Right now , so the gradient sensitivity , ensuring that the noise scale matches the gradient sensitivity, through the formula: ； Get the adaptive cross entropy loss gradient with adaptive noise protection ; in, is the local cross entropy loss of client i with respect to the model parameters The original gradient of is the failure probability, is the identity matrix, is Gaussian noise, To adjust the privacy budget.

6. A privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for verifying gradient compliance is: Client generates zero-knowledge proof , is the adaptive cross entropy loss gradient, For the degree matrix, prove the gradient legality and noise compliance; Generate zero-knowledge proof by the client to verify whether the adaptive cross entropy loss gradient is determined by local data Correct calculation; verify that the added noise meets distribution, and: ； in, is the gradient clipping threshold, is the failure probability, The privacy budget is adjusted dynamically.

7. The privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for performing security aggregation is: If the gradient legitimacy and noise compliance are verified, the verified gradients are securely aggregated using the formula: ； Calculate the safety aggregation gradient ; in, is the adaptive cross entropy loss gradient, is the total number of gradients that pass the verification, To verify the gradient index, It is an indicator function, which is 1 if the verification is successful, and 0 otherwise.

8. The privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for optimizing the meta-model is: After security aggregation is completed, a meta-model is constructed and optimized using the knowledge distillation loss function. Building a lightweight neural network meta-model , input original features and client meta features , output the prediction result of integrating global knowledge, through the knowledge distillation loss function formula: ； Perform knowledge distillation to obtain an optimized meta-model; in, is the total number of client meta-features, is the client meta-feature index, is the KL divergence, is the current global model, is the global model parameter, are the learnable parameters of the meta-model.

9. The privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for identifying knowledge leakage risks is: Generate adversarial samples through the fast gradient sign method FGSM, through the formula: ； Get adversarial examples ; in, is the disturbance intensity, is the loss function, is the loss function The gradient of the original sample x is: is the true label corresponding to the original sample x, is a symbolic function, is the global model parameter, is the current global model; Based on the adversarial samples , through the formula: ； The leakage risk value R is calculated. A high leakage risk value R indicates that the model is more sensitive to adversarial samples and there is a risk of knowledge leakage. in, is the total number of samples, is the sample index, is the confidence of the model in the prediction results, For adversarial samples, For the original sample, For the A global model of adversarial examples, For the A global model of adversarial examples, is the global model parameter; Identify knowledge leakage risks based on leakage risk values.

10. The privacy-preserving data joint modeling method based on federated learning according to claim 1, characterized in that: The method for performing initialization training is: By formula Get the noise intensity ; in, is the neuronal sensitivity, is the neuron index, is the basic noise standard deviation, To adjust the parameters, is the maximum value of all neuron sensitivities; In the metamodel Directional noise is injected into Calculate the adjustment parameters after injecting directional noise into the initial parameters ,in, are the metamodel parameters of neuron j, is directional noise; Noise was injected into the neurons in the high-sensitivity group, while the others remained unchanged; Obtain client meta-features and historical training performance to dynamically generate personalized initial parameters; Generate a function with initial parameters: ； Get the initial parameters ; in, is the neural network meta-model, is the four-dimensional element feature vector of client i, is the adjustment parameter after injecting directional noise, is the historical training gradient accumulation, is the learning rate parameter; Initialize training based on initial parameters.

Images