Cross-college practical training data sharing method based on data privacy

Abstract

The invention discloses a cross-college practical training data sharing method based on data privacy, and relates to the technical field of data sharing. Colleges participating in data sharing train models on local data and add differential privacy noise in gradient updating; generating a soft label for the local model as uploading information and uploading the uploading information to a server communicating with each college; the server dynamically weights and aggregates the soft labels according to the received data volume of the soft labels, the local verification accuracy and the gradient norm to obtain global soft labels; and the server uses the global soft label to train a global model and issues the global model to each college. The colleges participating in data sharing only need to adopt local practical training data to train a local model, and after training is completed, the local model capable of representing practical training data features is utilized to generate a soft label to participate in later global training and final training data sharing. Therefore, each college participating in data sharing does not need to directly use original practical training data, and safe sharing of large-scale practical training data of different colleges can be completed.

Description

Technical Field

[0001] This application relates to the field of data sharing technology, specifically to a cross-institutional training data sharing method based on data privacy. Background Technology

[0002] With the deep penetration of artificial intelligence and big data technologies into higher education, sharing high-quality training data across institutions has become a key requirement for cultivating interdisciplinary engineering talents. However, institutional data is inherently sensitive to privacy and heterogeneous due to its diverse sources. Directly aggregating data for training not only faces compliance risks but is also limited by technical bottlenecks such as insufficient sample size and sparse feature dimensions caused by "data silos." Federated Learning (FL), as a distributed framework where the data remains stationary while the model moves, provides a feasible path for cross-domain collaborative modeling.

[0003] While the FL (Frequency Federation) approach has made progress in privacy compliance, its direct application to cross-institutional training scenarios still reveals three shortcomings. First, existing patents generally assume that the data distribution of participating parties is independent and identically distributed (IID), but differences in institutional majors lead to data exhibiting multi-layered non-independent and identically distributed (Non-IID) characteristics across "majors-courses-training tasks." Second, educational data contains a large amount of time-series training logs and multimodal project files. Traditional horizontal federation only aligns samples and cannot solve the problem of heterogeneous feature spaces. While vertical federation can align features, it requires a trusted third party for entity alignment, contradicting the minimum trust assumption of the campus. Third, mainstream solutions use a server-client star topology, with all gradients in plaintext aggregated at the central node. Once the server is compromised, sensitive information such as student grades and behavioral trajectories can be inferred through gradient inversion.

[0004] To address the aforementioned shortcomings, recent research has begun to focus on a hybrid paradigm of hierarchical federation and knowledge distillation. Among these, the hybrid federated learning framework decouples vertical feature alignment from horizontal sample alignment, reducing communication overhead through histogram approximation and gRPC streaming compression. However, this approach still relies on a single central aggregation node and fails to address the personalized evaluation problem in Non-IID scenarios. The tree-based broadcast synchronization algorithm attempts to replace the star topology with a complete binary tree topology, splitting gradients into sub-files for parallel distribution to alleviate single-point bottlenecks. However, this algorithm only focuses on model file synchronization and does not address the multi-layered Non-IID correction mechanism unique to educational data. Chinese patent CN120705235B discloses a federated governance method and system for multi-source vocational education data, which for the first time dynamically allocates differential privacy budgets according to professional scale, which can suppress excessive noise in small-sample professional data to some extent. However, it lacks alignment strategies for heterogeneous feature spaces across institutions, resulting in the ineffective fusion of features related to practical training tasks.

[0005] In summary, existing horizontal / vertical federated learning has formed a relatively complete privacy protection framework. However, when faced with high-dimensional heterogeneity, multi-layered Non-IID, and low-trust environments of cross-institutional training data, problems such as low global model accuracy and high risk of single-point leakage still exist. Therefore, how to achieve heterogeneous feature alignment to support the secure sharing of large-scale cross-institutional training data without requiring the original data to leave the country or relying on trusted third parties is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, this application proposes the following technical solution: In a first aspect, embodiments of this application provide a method for sharing cross-institutional training data based on data privacy, including: Each participating institution trains its model on local data and adds differential privacy noise to the gradient update; Generate soft tags for the local model as upload information and upload them to the server that communicates with various universities; The server dynamically weights and aggregates the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags; The server uses the global soft labels to train a global model and distributes it to various institutions.

[0007] In one possible implementation, the participating institutions train a model on local data and add differential privacy noise in gradient updates, including: Each institution preprocesses its local training data and aligns the features using a feature alignment algorithm; The model is trained locally based on preprocessed local training data, and differential privacy noise is added during parameter updates.

[0008] In one possible implementation, the institutions preprocess their local training data and perform feature alignment using a feature alignment algorithm, including: Each institution first cleaned its local training data, improving the basic quality by imputing missing values, removing outliers, and deduplicating duplicate records. Then, Z-score standardization was performed on the numerical fields to bring the mean to zero and the variance to one, while one-hot or target coding was used for the categorical fields to ensure consistency of units. Next, the feature alignment algorithm is started to estimate the kernel density distribution of the common reference samples. Broadcast to all schools; Local construction reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. ; After training, each institution inputs the original features one by one into T to obtain aligned representations, forming a cleaned and standardized aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training.

[0009] In one possible implementation, the model training based on preprocessed local training data and the addition of differential privacy noise during parameter updates include: The preprocessed local training data is divided into mini-batches, and mini-batches of data are randomly selected in each training epoch for model training. For each mini-batch of data, the gradient of the model parameters is calculated, and gradient clipping is used to limit the sensitivity of the gradient in order to control the influence of individual samples on the gradient. Then, based on the preset privacy budget, the standard deviation of Gaussian noise that meets the differential privacy requirements is calculated, Gaussian noise of corresponding intensity is generated and added to the clipped gradient; Finally, the model parameters are updated using the noisy gradient to complete local training.

[0010] In one possible implementation, the objective function for local training is: in: Let the local model parameter weights of the i-th institution be defined in the t-th round of communication. For local loss function, For learning rate, For vector differential operators, To satisfy the noise variance of differential privacy, The identity matrix used to generate differential privacy noise, This represents the calculation of a normal distribution.

[0011] In one possible implementation, generating soft tags for the local model as upload information and uploading them to the server that communicates with various institutions includes: After completing local differential privacy training, each institution generates soft labels for the trained local model: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. For public datasets; The soft tag, along with the local verification accuracy, data volume, and gradient norm, is packaged into an encrypted message and uploaded to the server via a two-way mTLS channel, completing the knowledge distillation process.

[0012] In one possible implementation, the server dynamically weights and aggregates the soft tags based on the amount of received soft tag data, local verification accuracy, and gradient norm to obtain global soft tags, including: The comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution. ,in: For data volume, To verify accuracy locally, For gradient norm, This is the data volume weighting coefficient. This is a weighting factor for the accuracy of local validation. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization; The dynamic weight is obtained by normalizing the comprehensive contribution score: , This represents the sum of the overall contribution scores of all institutions. Then, using this weight, a weighted average is aggregated for the soft tags of all institutions to generate a global soft tag: .

[0013] In one possible implementation, the server uses the global soft labels to train a global model and distributes it to various institutions, including: The server uses the aggregated global soft tags to train a global model. The loss function is: in: and To balance the weights, For cross-entropy loss, Kullback-Leibler divergence measures the difference between two probability distributions; To compensate for the scaling effect, The true label vectors of samples in the public dataset. For global model of samples The raw logits output; The server will use the trained global model The data was distributed to various institutions, and Shapley values ​​were calculated to assess their contributions.

[0014] In one possible implementation, the server will use the trained global model The information was distributed to various institutions, and Shapley scores were calculated to assess their contributions, including: in: For colleges and universities The precise Shapley value measures its expected marginal contribution. For without Any subset of the alliance, For all colleges and universities, For the number of members of Alliance S, The total number of all institutions For factorial operator, The probability weight for the emergence of this alliance. For utility function, Indicates that the colleges and universities Joining the alliance as a whole, For colleges and universities Against the alliance The marginal utility increment.

[0015] Secondly, embodiments of this application provide a cross-institutional training data sharing system based on data privacy, including: The data processing module is used by participating institutions to train models on local data and add differential privacy noise in gradient updates. The information upload module is used to generate soft tags for the local model as upload information and upload them to the server that communicates with various universities; The data aggregation module is used by the server to dynamically weight and aggregate the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags; The data sharing module is used by the server to train a global model using the global soft labels and distribute it to various institutions.

[0016] Thirdly, embodiments of this application provide a training device, including: processor; Memory; And a computer program, wherein the computer program is stored in the memory, the computer program including instructions that, when executed by the processor, cause the training device to participate in the method described in any possible implementation of the first aspect.

[0017] In this embodiment, participating institutions only need to train their local models using local training data. After training, they use the local models, which represent the characteristics of the training data, to generate soft labels for later global training and final training data sharing. This way, each participating institution does not need to directly use the original training data to achieve secure sharing of large-scale training data across different institutions. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a cross-institutional training data sharing method based on data privacy, provided for an embodiment of this application; Figure 2 This application provides a data sharing implementation architecture diagram. Figure 3 A schematic diagram of a cross-institutional training data sharing system based on data privacy, provided for an embodiment of this application; Figure 4 This is a schematic diagram of a training device provided in an embodiment of this application. Detailed Implementation

[0019] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0020] See Figure 1 The cross-institutional training data sharing method based on data privacy provided in this embodiment includes: S101, participating institutions train models on local data and add differential privacy noise in gradient updates.

[0021] See Figure 2 In this embodiment, multiple institutions participate in the sharing of training data, with each institution possessing multiple training devices used to collect students' training data. When an institution decides to participate in sharing training data with multiple other institutions, it designates one training device as the data uploader, and the other training devices transmit the training data to that designated device. Each institution's designated training device establishes a communication connection with the server responsible for collecting the data uploaded by each institution.

[0022] After the participating institutions establish communication with the server, each institution preprocesses its local training data and aligns its features using a feature alignment algorithm. Based on the preprocessed local training data, the institution trains the model locally and adds differential privacy noise during parameter updates.

[0023] Specifically, each institution first cleans its local training data, improving basic quality through missing value imputation, outlier removal, and duplicate record deduplication. Then, numerical fields are Z-score standardized to zero mean and one variance, while categorical fields are one-hot or target encoding to ensure dimensional consistency. Next, the feature alignment algorithm is initiated: the server estimates the distribution of the kernel density of the common reference samples. Broadcast to various universities, local structure reversible Transform the network T, using Wasserstein-1 distance as the loss, and train T using gradient-penalized WGAN-GP until the local distribution is reached. and The difference in advance measurement is less than a set threshold. After training, each institution inputs the original features one by one into T to obtain aligned representations, forming a cleaned, standardized, and aligned dataset with the same dimension and distribution as the global feature space, providing a unified input for subsequent federated training.

[0024] The process by which universities train models on local data and add differential privacy noise to parameter updates is as follows: First, each university cleans and preprocesses its local data to ensure data quality. Then, the data is divided into mini-batches, and a mini-batch is randomly selected for model training in each training epoch. Next, for each mini-batch, the gradient of the model parameters is calculated, and gradient clipping is used to limit the sensitivity of the gradient to control the influence of a single sample on the gradient. Subsequently, based on a preset privacy budget, the standard deviation of Gaussian noise that meets the differential privacy requirements is calculated, and Gaussian noise of appropriate intensity is generated and added to the clipped gradient. Finally, the model parameters are updated using the noisy gradient, completing local training. Throughout the process, universities only share the noisy model parameters or aggregated results; the original data remains locally, thus achieving collaborative model training while ensuring data privacy.

[0025] The following is the objective function for local training: in: Let the local model parameter weights of the i-th institution be defined in the t-th round of communication. For local loss function, For learning rate, For vector differential operators, To satisfy the noise variance of differential privacy, The identity matrix used to generate differential privacy noise, This represents the calculation of a normal distribution.

[0026] S102 generates soft tags for the local model as upload information and uploads them to the server for communication with various universities.

[0027] After completing differential privacy training locally, each institution sets the current model as a teacher network, sequentially infers forward from the public unlabeled sample set uniformly distributed by the server, obtains logits, softens them by softening the distribution parameters, and then generates soft labels in the form of probability distribution through softmax normalization.

[0028] The specific implementation process is as follows: ,in: For normalization function, For local models, For public unlabeled samples, These are the parameters used for softening the distribution. It is a public dataset.

[0029] Subsequently, FP16 quantization was used to compress the volume, and the soft tag, along with the local verification accuracy, data volume, and gradient norm, was packaged into an encrypted message and uploaded to the server through a bidirectional mTLS channel, completing a knowledge upload communication that does not require sharing the original data.

[0030] S103, the server dynamically weights and aggregates the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags.

[0031] First, the comprehensive contribution score for each institution is calculated using the local validation accuracy, data volume, gradient norm, and corresponding weighting coefficients uploaded by each institution: ,in: For data volume, To verify accuracy locally, For gradient norm, This is the data volume weighting coefficient. This is a weighting factor for the accuracy of local validation. These are the gradient norm weighting coefficients. Let i be the number of local samples for institution i. The total number of samples from all institutions. This is a scaling factor used to map the gradient norm to the sigmoid-sensitive region. This indicates gradient norm normalization.

[0032] Then, the comprehensive contribution score is normalized to obtain the dynamic weight: , This represents the sum of the overall contribution scores of all institutions. Subsequently, a weighted average is applied to the soft tags of all institutions using this weight to generate a global soft tag: .

[0033] S104, the server uses the global soft label to train a global model and distributes it to each institution.

[0034] The server uses the aggregated global soft tags to train a global model. The loss function is: in: and To balance the weights, For cross-entropy loss, Kullback-Leibler divergence measures the difference between two probability distributions; To compensate for the scaling effect, The true label vectors of samples in the public dataset. For global model of samples The raw logits output.

[0035] The server will use the trained global model The data was distributed to all institutions, and a Shapley score was calculated to assess their contributions. Incentives were then awarded based on the Shapley score used to evaluate institutional contributions. Specifically: in: For colleges and universities The precise Shapley value measures its expected marginal contribution. For without Any subset of the alliance, For all colleges and universities, For the number of members of Alliance S, The total number of all institutions For factorial operator, The probability weight for the emergence of this alliance. For utility function, Indicates that the colleges and universities Joining the alliance as a whole, For colleges and universities Against the alliance The marginal utility increment.

[0036] Corresponding to the data privacy-based cross-institutional training data sharing method provided by the above-mentioned strength, this application also provides an embodiment of a data privacy-based cross-institutional training data sharing system.

[0037] See Figure 3 The cross-institutional training data sharing system 20 based on data privacy in this embodiment includes: The data processing module 201 is used by participating institutions to train models on local data and add differential privacy noise in gradient updates.

[0038] The information upload module 202 is used to generate soft tags for the local model as upload information and upload them to the server that communicates with various colleges and universities.

[0039] The data aggregation module 203 is used by the server to dynamically weight and aggregate the soft tags based on the amount of data received, the local verification accuracy, and the gradient norm to obtain global soft tags.

[0040] The data sharing module 204 is used by the server to train a global model using the global soft labels and distribute it to various institutions.

[0041] Corresponding to the above embodiments, this application also provides a training device.

[0042] See Figure 4 The training device 300 in this embodiment includes a processor 301, a memory 302, and a communication unit 303. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the training device shown in the figure does not constitute a limitation on the embodiments of this application. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0043] The communication unit 303 is used to establish a communication channel, so that the training equipment can communicate with other local training equipment and servers.

[0044] The processor 301 serves as the control center of the training equipment. It connects various parts of the equipment via interfaces and lines, and executes software programs and / or modules stored in the memory 302, as well as calling data stored in the memory, to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 301 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0045] Memory 302 is used to store the execution instructions of processor 301. Memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0046] When the execution instructions in memory 302 are executed by processor 301, the training device 300 is able to participate in data sharing in the above method embodiments.

[0047] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0048] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A data privacy-based cross-institutional practical training data sharing method, characterized in that, The method comprises the following steps: Each school participating in data sharing trains a model on local data and adds differential privacy noise in gradient update; Soft labels are generated for the local model as upload information and uploaded to a server in communication with each school; The server dynamically weights and aggregates the soft labels according to the data volume, local verification accuracy and gradient norm of the received soft labels to obtain global soft labels; The server trains a global model using the global soft labels and distributes the global model to each school.

2. The data privacy based cross-institutional training data sharing method according to claim 1, characterized in that, The method that each school participating in data sharing trains a model on local data and adds differential privacy noise in gradient update comprises the following steps: Each school pre-processes local training data and aligns features through a feature alignment algorithm; Based on the pre-processed local training data, a model is trained locally, and differential privacy noise is added in parameter update.

3. The data privacy based cross-institutional training data sharing method according to claim 2, characterized in that, The method that each school pre-processes local training data and aligns features through a feature alignment algorithm comprises the following steps: Each school first cleans the local training data, and completes basic quality improvement through missing value imputation, outlier removal and duplicate record deduplication; Then, the Z-score standardization is performed on the numerical fields to make the mean zero and the variance one, and the one-hot or target encoding is performed on the category fields to ensure the consistency of the dimension; Next, a feature alignment algorithm is initiated to align the kernel density estimation distributions of the common reference samples Broadcast to each college; Locally constructed invertible Transform network T, train T by gradient-penalized WGAN-GP with Wasserstein-1 distance as loss until local distribution With Pre-push measure difference is less than a set threshold ; After training, each school inputs the original features into T one by one to obtain aligned representations, and forms a cleaned and standardized aligned dataset with the same dimension and distribution as the global feature space, thereby providing a unified input for subsequent federated training.

4. The data privacy based cross-institutional training data sharing method according to claim 2, wherein, The method that each school trains a model based on the pre-processed local training data locally and adds differential privacy noise in parameter update comprises the following steps: The pre-processed local training data is divided into small batches, and small batch data is randomly extracted for model training in each training round (epoch); For each small batch of data, the gradient of the model parameters is calculated, and the gradient clipping technique is used to limit the sensitivity of the gradient to control the influence of a single sample on the gradient; Then, according to a preset privacy budget, the standard deviation of the Gaussian noise satisfying the differential privacy requirement is calculated, the Gaussian noise with the corresponding intensity is generated and added to the clipped gradient; Finally, the model parameters are updated using the noisy gradient to complete local training.

5. The data privacy based cross-institutional practical training data sharing method according to claim 4, characterized in that, The objective function of local training is: where: is the local model parameter weight of the ith institute at the tth round of communication, is the local loss function, is the learning rate, is the vector differential operator, is the noise variance satisfying differential privacy, is the identity matrix used to generate the differential privacy noise, denotes the normal distribution calculation.

6. The data privacy based cross-institutional training data sharing method according to claim 1, wherein, The method that soft labels are generated for the local model as upload information and uploaded to a server in communication with each school comprises the following steps: After completing the local differential privacy training, the colleges and universities generate soft labels for the trained local models: wherein: is a normalization function, is a local model, is a public unlabeled sample, is a parameter for softening the distribution, is a public dataset; The soft labels are packaged into an encrypted message together with the local verification accuracy, data volume and gradient norm, and uploaded to the server through a bidirectional mTLS channel to complete the upload of the knowledge distillation link.

7. The data privacy based cross-institutional training data sharing method according to claim 6, characterized in that, The method that the server dynamically weights and aggregates the soft labels according to the data volume, local verification accuracy and gradient norm of the received soft labels to obtain global soft labels comprises the following steps: The comprehensive contribution score of each college is calculated by using the received local verification accuracy, data volume and gradient norm and corresponding weight coefficients uploaded by each college: wherein: is the data volume, is the local verification accuracy, is the gradient norm, is the data volume weight coefficient, is the local verification accuracy weight coefficient, is the gradient norm weight coefficient, is the number of local samples of the college i, is the total number of samples of all colleges, is the scaling factor for mapping the gradient norm to the sigmoid sensitive interval, denotes the gradient norm normalization; normalizing the overall contribution score to obtain a dynamic weight: , denotes the sum of the overall contribution scores of all colleges Subsequently, the soft labels of all colleges are weighted and averaged to generate a global soft label: .

8. The data privacy based cross-institutional training data sharing method according to claim 7, characterized in that, The method that the server trains a global model using the global soft labels and distributes the global model to each school comprises the following steps: The server trains a global model using the aggregated global soft labels The loss function is: where: and is a balancing weight, is a cross-entropy loss, is a Kullback-Leibler divergence, measuring the difference between two probability distributions; is a compensation scaling effect, is a true label vector of the public dataset samples, is the original logits output by the global model for a sample ​ The server will use the trained global model The data was distributed to various institutions, and Shapley values ​​were calculated to assess their contributions.

9. The data privacy based cross-institutional training data sharing method according to claim 8, characterized in that, The server will train the global model Distributed to each college, and calculate the Shapley value to evaluate its contribution, including: where: is the precise Shapley value for a school, measuring its expected marginal contribution, is the marginal utility of school to any coalition subset is the full set of schools, is the number of members of coalition S, is the number of all schools, is the factorial operator, is the probability weight of the coalition occurring, is the utility function, denotes the addition of school to the coalition as a whole, is the marginal utility increment of school to coalition .

10. A data privacy-based cross-institutional practical training data sharing system, characterized in that, The method comprises the following steps: A data processing module is configured to train a model on local data by each school participating in data sharing and add differential privacy noise in gradient update; An information upload module is configured to generate soft labels for a local model as upload information and upload the soft labels to a server in communication with each school. A data aggregation module is configured to aggregate the soft labels to obtain global soft labels by the server according to the data amount of the received soft labels, the local verification accuracy and the gradient norm dynamic weighting. A data sharing module is configured to train a global model by the server using the global soft labels and distribute the global model to each college.

Images