A data privacy-based cross-institutional practical training data sharing method

Abstract

The application discloses a data privacy-based cross-college practical training data sharing method, relates to the technical field of data sharing, and involves the following steps: each college participating in data sharing trains a model on local data and adds differential privacy noise in gradient updating; soft labels of the local model are generated as upload information and uploaded to a server in communication with each college; the server dynamically weights and aggregates the soft labels to obtain global soft labels according to the data amount of the received soft labels, a local verification accuracy and a gradient norm; and the server trains a global model using the global soft labels and distributes the global model to each college. Each college participating in data sharing only needs to train a local model using local practical training data, generates soft labels of the local model capable of representing characteristics of the practical training data after the training is completed, and participates in subsequent global training and final training data sharing. In this way, each college participating in data sharing can complete safe sharing of large-scale practical training data of different colleges without directly using original practical training data.

Description

Technical Field

[0001] This application relates to the field of data sharing technology, specifically to a cross-institutional training data sharing method based on data privacy. Background Technology

[0002] With the deep penetration of artificial intelligence and big data technologies into higher education, sharing high-quality training data across institutions has become a key requirement for cultivating interdisciplinary engineering talents. However, institutional data is inherently sensitive to privacy and heterogeneous due to its diverse sources. Directly aggregating data for training not only faces compliance risks but is also limited by technical bottlenecks such as insufficient sample size and sparse feature dimensions caused by "data silos." Federated Learning (FL), as a distributed framework where the data remains stationary while the model moves, provides a feasible path for cross-domain collaborative modeling.

[0003] While the FL (Frequency Federation) approach has made progress in privacy compliance, its direct application to cross-institutional training scenarios still reveals three shortcomings. First, existing patents generally assume that the data distribution of participating parties is independent and identically distributed (IID), but differences in institutional majors lead to data exhibiting multi-layered non-independent and identically distributed (Non-IID) characteristics across "majors-courses-training tasks." Second, educational data contains a large amount of time-series training logs and multimodal project files. Traditional horizontal federation only aligns samples and cannot solve the problem of heterogeneous feature spaces. While vertical federation can align features, it requires a trusted third party for entity alignment, contradicting the minimum trust assumption of the campus. Third, mainstream solutions use a server-client star topology, with all gradients in plaintext aggregated at the central node. Once the server is compromised, sensitive information such as student grades and behavioral trajectories can be inferred through gradient inversion.

[0004] To address the aforementioned shortcomings, recent research has begun to focus on a hybrid paradigm of hierarchical federation and knowledge distillation. Among these, the hybrid federated learning framework decouples vertical feature alignment from horizontal sample alignment, reducing communication overhead through histogram approximation and gRPC streaming compression. However, this approach still relies on a single central aggregation node and fails to address the personalized evaluation problem in Non-IID scenarios. The tree-based broadcast synchronization algorithm attempts to replace the star topology with a complete binary tree topology, splitting gradients into sub-files for parallel distribution to alleviate single-point bottlenecks. However, this algorithm only focuses on model file synchronization and does not address the multi-layered Non-IID correction mechanism unique to educational data. Chinese patent CN120705235B discloses a federated governance method and system for multi-source vocational education data, which for the first time dynamically allocates differential privacy budgets according to professional scale, which can suppress excessive noise in small-sample professional data to some extent. However, it lacks alignment strategies for heterogeneous feature spaces across institutions, resulting in the ineffective fusion of features related to practical training tasks.

[0005] In summary, existing horizontal / vertical federated learning has formed a relatively complete privacy protection framework. However, when faced with high-dimensional heterogeneity, multi-layered Non-IID, and low-trust environments of cross-institutional training data, problems such as low global model accuracy and high risk of single-point leakage still exist. Therefore, how to achieve heterogeneous feature alignment to support the secure sharing of large-scale cross-institutional training data without requiring the original data to leave the country or relying on trusted third parties is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, this application proposes the following technical solution:

[0007] In a first aspect, embodiments of this application provide a method for sharing cross-institutional training data based on data privacy, including:

[0008] Each participating institution trains its model on local data and adds differential privacy noise to the gradient update;

[0009] Generate soft tags for the local model as upload information and upload them to the server that communicates with various universities;

[0010] The server dynamically weights and aggregates the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags;

[0011] The server uses the global soft labels to train a global model and distributes it to various institutions.

[0012] In one possible implementation, the participating institutions train a model on local data and add differential privacy noise in gradient updates, including:

[0013] Each institution preprocesses its local training data and aligns the features using a feature alignment algorithm;

[0014] The model is trained locally based on preprocessed local training data, and differential privacy noise is added during parameter updates.

[0015] In one possible implementation, the institutions preprocess their local training data and perform feature alignment using a feature alignment algorithm, including:

[0016] Each institution first cleaned its local training data, improving the basic quality by imputing missing values, removing outliers, and deduplicating duplicate records.

[0017] Then, Z-score standardization was performed on the numerical fields to bring the mean to zero and the variance to one, while one-hot or target coding was used for the categorical fields to ensure consistency of units.

[0018] Next, the feature alignment algorithm is started to estimate the kernel density distribution of the common reference samples. Broadcast to all schools;

[0019] Local construction reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. ;

[0020] After training, each institution inputs the original features one by one into T to obtain aligned representations, forming a cleaned and standardized aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training.

[0021] In one possible implementation, the model training based on preprocessed local training data and the addition of differential privacy noise during parameter updates include:

[0022] The preprocessed local training data is divided into mini-batches, and mini-batches of data are randomly selected in each training epoch for model training.

[0023] For each mini-batch of data, the gradient of the model parameters is calculated, and gradient clipping is used to limit the sensitivity of the gradient in order to control the influence of individual samples on the gradient.

[0024] Then, based on the preset privacy budget, the standard deviation of Gaussian noise that meets the differential privacy requirements is calculated, Gaussian noise of corresponding intensity is generated and added to the clipped gradient;

[0025] Finally, the model parameters are updated using the noisy gradient to complete local training.

[0026] In one possible implementation, the objective function for local training is:

[0027]

[0028] in: Let the local model parameter weights of the i-th institution be defined in the t-th round of communication. For local loss function, For learning rate, For vector differential operators, To satisfy the noise variance of differential privacy, The identity matrix used to generate differential privacy noise, This represents the calculation of a normal distribution.

[0029] In one possible implementation, generating soft tags for the local model as upload information and uploading them to the server that communicates with various institutions includes:

[0030] After completing local differential privacy training, each institution generates soft labels for the trained local model: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. For public datasets;

[0031] The soft tag, along with the local verification accuracy, data volume, and gradient norm, is packaged into an encrypted message and uploaded to the server via a two-way mTLS channel, completing the knowledge distillation process.

[0032] In one possible implementation, the server dynamically weights and aggregates the soft tags based on the amount of received soft tag data, local verification accuracy, and gradient norm to obtain global soft tags, including:

[0033] The comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution. ,in: For data volume, To verify accuracy locally, For gradient norm, The data volume weighting coefficient. This is a weighting coefficient for local validation accuracy. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization;

[0034] The dynamic weight is obtained by normalizing the comprehensive contribution score: , This represents the sum of the overall contribution scores of all institutions.

[0035] Then, using this weight, a weighted average is aggregated for the soft tags of all institutions to generate a global soft tag: .

[0036] In one possible implementation, the server uses the global soft labels to train a global model and distributes it to various institutions, including:

[0037] The server uses the aggregated global soft tags to train a global model. The loss function is:

[0038]

[0039] in: and To balance the weights, For cross-entropy loss, Kullback-Leibler divergence measures the difference between two probability distributions; To compensate for the scaling effect, The true label vectors of samples in the public dataset. For global model of samples The raw logits output;

[0040] The server will use the trained global model The data was distributed to various institutions, and Shapley values ​​were calculated to assess their contributions.

[0041] In one possible implementation, the server will use the trained global model The information was distributed to various institutions, and Shapley scores were calculated to assess their contributions, including:

[0042]

[0043] in: For colleges and universities The precise Shapley value measures its expected marginal contribution. For without Any subset of the alliance, For all colleges and universities, For the number of members of Alliance S, The total number of all institutions For factorial operator, The probability weight for the emergence of this alliance. For utility function, Indicates that the colleges and universities Joining the alliance as a whole, For colleges and universities Against the alliance The marginal utility increment.

[0044] Secondly, embodiments of this application provide a cross-institutional training data sharing system based on data privacy, including:

[0045] The data processing module is used by participating institutions to train models on local data and add differential privacy noise in gradient updates.

[0046] The information upload module is used to generate soft tags for the local model as upload information and upload them to the server that communicates with various universities;

[0047] The data aggregation module is used by the server to dynamically weight and aggregate the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags;

[0048] The data sharing module is used by the server to train a global model using the global soft labels and distribute it to various institutions.

[0049] Thirdly, embodiments of this application provide a training device, including:

[0050] processor;

[0051] Memory;

[0052] And a computer program, wherein the computer program is stored in the memory, the computer program including instructions that, when executed by the processor, cause the training device to participate in the method described in any possible implementation of the first aspect.

[0053] In this embodiment, participating institutions only need to train their local models using local training data. After training, they use the local models, which represent the characteristics of the training data, to generate soft labels for later global training and final training data sharing. This way, each participating institution does not need to directly use the original training data to achieve secure sharing of large-scale training data across different institutions. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a cross-institutional training data sharing method based on data privacy, provided for an embodiment of this application;

[0055] Figure 2 A data sharing implementation architecture diagram provided in this application embodiment;

[0056] Figure 3 A schematic diagram of a cross-institutional training data sharing system based on data privacy, provided for an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of a training device provided in an embodiment of this application. Detailed Implementation

[0058] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0059] See Figure 1The cross-institutional training data sharing method based on data privacy provided in this embodiment includes:

[0060] S101, participating institutions train models on local data and add differential privacy noise in gradient updates.

[0061] See Figure 2 In this embodiment, multiple institutions participate in the sharing of training data, with each institution possessing multiple training devices used to collect students' training data. When an institution decides to participate in sharing training data with multiple other institutions, it designates one training device as the data uploader, and the other training devices transmit the training data to that designated device. Each institution's designated training device establishes a communication connection with the server responsible for collecting the data uploaded by each institution.

[0062] After the participating institutions establish communication with the server, each institution preprocesses its local training data and aligns its features using a feature alignment algorithm. Based on the preprocessed local training data, the institution trains the model locally and adds differential privacy noise during parameter updates.

[0063] Specifically, each institution first cleans its local training data, improving basic quality through missing value imputation, outlier removal, and duplicate record deduplication. Then, numerical fields are Z-score standardized to zero mean and one variance, while categorical fields are one-hot or target encoding to ensure dimensional consistency. Next, the feature alignment algorithm is initiated: the server estimates the distribution of the kernel density of the common reference samples. Broadcast to various universities, local structure reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. After training, each institution inputs the original features one by one into T to obtain aligned representations, forming a cleaned, standardized, and aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training.

[0064] The process by which universities train models on local data and add differential privacy noise to parameter updates is as follows: First, each university cleans and preprocesses its local data to ensure data quality. Then, the data is divided into mini-batches, and a mini-batch is randomly selected for model training in each training epoch. Next, for each mini-batch, the gradient of the model parameters is calculated, and gradient clipping is used to limit the sensitivity of the gradient to control the influence of a single sample on the gradient. Subsequently, based on a preset privacy budget, the standard deviation of Gaussian noise that meets the differential privacy requirements is calculated, and Gaussian noise of appropriate intensity is generated and added to the clipped gradient. Finally, the model parameters are updated using the noisy gradient, completing local training. Throughout the process, universities only share the noisy model parameters or aggregated results; the original data remains locally, thus achieving collaborative model training while ensuring data privacy.

[0065] The following is the objective function for local training:

[0066]

[0067] in: Let the local model parameter weights of the i-th institution be defined in the t-th round of communication. For local loss function, For learning rate, For vector differential operators, To satisfy the noise variance of differential privacy, The identity matrix used to generate differential privacy noise, This represents the calculation of a normal distribution.

[0068] S102 generates soft tags for the local model as upload information and uploads them to the server for communication with various universities.

[0069] After completing differential privacy training locally, each institution sets the current model as a teacher network, sequentially infers forward from the public unlabeled sample set uniformly distributed by the server, obtains logits, softens them by softening the distribution parameters, and then generates soft labels in the form of probability distribution through softmax normalization.

[0070] The specific implementation process is as follows: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. It is a public dataset.

[0071] Subsequently, FP16 quantization was used to compress the volume, and the soft tag, along with the local verification accuracy, data volume, and gradient norm, was packaged into an encrypted message and uploaded to the server through a bidirectional mTLS channel, completing a knowledge upload communication that does not require sharing the original data.

[0072] S103, the server dynamically weights and aggregates the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags.

[0073] First, the comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution: ,in: For data volume, To verify accuracy locally, For gradient norm, The data volume weighting coefficient. This is a weighting coefficient for local validation accuracy. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization.

[0074] Then, the comprehensive contribution score is normalized to obtain the dynamic weight: , This represents the sum of the overall contribution scores of all institutions. Subsequently, a weighted average is applied to the soft tags of all institutions using this weight to generate a global soft tag: .

[0075] S104, the server uses the global soft label to train a global model and distributes it to each institution.

[0076] The server uses the aggregated global soft tags to train a global model. The loss function is:

[0077]

[0078] in: and To balance the weights, For cross-entropy loss, Kullback-Leibler divergence measures the difference between two probability distributions; To compensate for the scaling effect, The true label vectors of samples in the public dataset. For global model of samples The raw logits output.

[0079] The server will use the trained global model The data is distributed to all institutions, and a Shapley score is calculated to assess their contribution. Incentives are then awarded based on the Shapley score used to evaluate institutional contributions. Specifically:

[0080]

[0081] in: For colleges and universities The precise Shapley value measures its expected marginal contribution. For without Any subset of the alliance, For all colleges and universities, For the number of members of Alliance S, The total number of all institutions For factorial operator, The probability weight for the emergence of this alliance. For utility function, Indicates that the colleges and universities Joining the alliance as a whole, For colleges and universities Against the alliance The marginal utility increment.

[0082] Corresponding to the data privacy-based cross-institutional training data sharing method provided by the above-mentioned strength, this application also provides an embodiment of a data privacy-based cross-institutional training data sharing system.

[0083] See Figure 3 The cross-institutional training data sharing system 20 based on data privacy in this embodiment includes:

[0084] The data processing module 201 is used by participating institutions to train models on local data and add differential privacy noise in gradient updates.

[0085] The information upload module 202 is used to generate soft tags for the local model as upload information and upload them to the server that communicates with various colleges and universities.

[0086] The data aggregation module 203 is used by the server to dynamically weight and aggregate the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags.

[0087] The data sharing module 204 is used by the server to train a global model using the global soft labels and distribute it to various institutions.

[0088] Corresponding to the above embodiments, this application also provides a training device.

[0089] See Figure 4 The training device 300 in this embodiment includes a processor 301, a memory 302, and a communication unit 303. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the training device shown in the figure does not constitute a limitation on the embodiments of this application. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0090] The communication unit 303 is used to establish a communication channel, so that the training equipment can communicate with other local training equipment and servers.

[0091] The processor 301 serves as the control center of the training equipment. It connects various parts of the equipment via interfaces and lines, and executes software programs and / or modules stored in the memory 302, as well as calling data stored in the memory, to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 301 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0092] Memory 302 is used to store the execution instructions of processor 301. Memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0093] When the execution instructions in memory 302 are executed by processor 301, the training device 300 is able to participate in data sharing in the above method embodiments.

[0094] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0095] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for sharing cross-institutional training data based on data privacy, characterized in that, include: Participating institutions train models on local data and add differential privacy noise to gradient updates, including: Each institution preprocesses its local training data and aligns the features using a feature alignment algorithm; The model is trained locally based on the preprocessed local training data, and differential privacy noise is added during parameter updates. Each institution preprocesses its local training data and performs feature alignment using a feature alignment algorithm, including: Each institution first cleaned its local training data, improving the basic quality by imputing missing values, removing outliers, and deduplicating duplicate records. Then, Z-score standardization was performed on the numerical fields to bring the mean to zero and the variance to one, while one-hot or target coding was used for the categorical fields to ensure consistency of units. Next, the feature alignment algorithm is started to estimate the kernel density distribution of the common reference samples. Broadcast to all schools; Local construction reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. ; After training, each institution inputs the original features one by one into T to obtain the aligned representation, forming a cleaned and standardized aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training; Generate soft tags for the local model as upload information and upload them to the server that communicates with various universities, including: After completing local differential privacy training, each institution generates soft labels for the trained local model: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. For public datasets; The soft tag, along with the local verification accuracy, data volume, and gradient norm, is packaged into an encrypted message and uploaded to the server via a two-way mTLS channel, completing the knowledge distillation process. The server dynamically weights and aggregates the received soft tags based on the data volume, local verification accuracy, and gradient norm to obtain global soft tags, including: The comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution. ,in: For data volume, To verify accuracy locally, For gradient norm, The data volume weighting coefficient. This is a weighting coefficient for local validation accuracy. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization; The dynamic weight is obtained by normalizing the comprehensive contribution score: , This represents the sum of the overall contribution scores of all institutions. Then, using this weight, a weighted average is aggregated for the soft tags of all institutions to generate a global soft tag: ; The server uses the global soft labels to train a global model and distributes it to various institutions.

2. The cross-institutional training data sharing method based on data privacy according to claim 1, characterized in that, The model is trained locally based on the preprocessed local training data, and differential privacy noise is added during parameter updates, including: The preprocessed local training data is divided into mini-batches, and mini-batches of data are randomly selected in each training epoch for model training. For each mini-batch of data, the gradient of the model parameters is calculated, and gradient clipping is used to limit the sensitivity of the gradient in order to control the influence of individual samples on the gradient. Then, based on the preset privacy budget, the standard deviation of Gaussian noise that meets the differential privacy requirements is calculated, Gaussian noise of corresponding intensity is generated and added to the clipped gradient; Finally, the model parameters are updated using the noisy gradient to complete local training.

3. The cross-institutional training data sharing method based on data privacy according to claim 2, characterized in that, The objective function for local training is: in: Let the local model parameter weights of the i-th institution be defined in the t-th round of communication. For local loss function, For learning rate, For vector differential operators, To satisfy the noise variance of differential privacy, The identity matrix used to generate differential privacy noise, This represents the calculation of a normal distribution.

4. The cross-institutional training data sharing method based on data privacy according to claim 1, characterized in that, The server uses the global soft labels to train a global model and distributes it to various institutions, including: The server uses the aggregated global soft tags to train a global model. The loss function is: in: and To balance the weights, For cross-entropy loss, Kullback-Leibler divergence measures the difference between two probability distributions; To compensate for the scaling effect, The true label vectors of samples in the public dataset. For global model of samples The raw logits output; The server will use the trained global model The data was distributed to various institutions, and Shapley values ​​were calculated to assess their contributions.

5. The cross-institutional training data sharing method based on data privacy according to claim 4, characterized in that, The server will use the trained global model The information was distributed to various institutions, and Shapley scores were calculated to assess their contributions, including: in: For colleges and universities The precise Shapley value measures its expected marginal contribution. For without Any subset of the alliance, For all colleges and universities, For the number of members of Alliance S, The total number of all institutions For factorial operator, The probability weight for the emergence of this alliance. For utility function, Indicates that the colleges and universities Joining the alliance as a whole, For colleges and universities Against the alliance The marginal utility increment.

6. A cross-institutional training data sharing system based on data privacy, characterized in that, include: The data processing module is used by participating institutions to train models on local data and add differential privacy noise in gradient updates, including: Each institution preprocesses its local training data and aligns the features using a feature alignment algorithm; The model is trained locally based on the preprocessed local training data, and differential privacy noise is added during parameter updates. Each institution preprocesses its local training data and performs feature alignment using a feature alignment algorithm, including: Each institution first cleaned its local training data, improving the basic quality by imputing missing values, removing outliers, and deduplicating duplicate records. Then, Z-score standardization was performed on the numerical fields to bring the mean to zero and the variance to one, while one-hot or target coding was used for the categorical fields to ensure consistency of units. Next, the feature alignment algorithm is started to estimate the kernel density distribution of the common reference samples. Broadcast to all schools; Local construction reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. ; After training, each institution inputs the original features one by one into T to obtain the aligned representation, forming a cleaned and standardized aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training; The information upload module is used to generate soft tags for the local model as upload information and upload them to the server that communicates with various universities, including: After completing local differential privacy training, each institution generates soft labels for the trained local model: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. For public datasets; The soft tag, along with the local verification accuracy, data volume, and gradient norm, is packaged into an encrypted message and uploaded to the server via a two-way mTLS channel, completing the knowledge distillation process. The data aggregation module is used by the server to dynamically weight and aggregate the soft tags based on the received soft tag data volume, local verification accuracy, and gradient norm to obtain global soft tags, including: The comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution. ,in: For data volume, To verify accuracy locally, For gradient norm, The data volume weighting coefficient. This is a weighting coefficient for local validation accuracy. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization; The dynamic weight is obtained by normalizing the comprehensive contribution score: , This represents the sum of the overall contribution scores of all institutions. Then, using this weight, a weighted average is aggregated for the soft tags of all institutions to generate a global soft tag: ; The data sharing module is used by the server to train a global model using the global soft labels and distribute it to various institutions.

Images