A privacy protection weakly supervised data labeling system based on federated learning
Patent Information
- Application Number
- CN202610843933.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-11
- Publication Date
- 2026-10-27
AI Technical Summary
1、本发明构建了一个完整的、端到端的隐私保护弱监督数据生成与标注闭环，弱监督信号的生成和初步标注均在客户端本地完成，数据安全得到最高级别的保障；
Smart Images
Figure CN122986497A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and more specifically, to a privacy-preserving, weakly supervised data annotation system based on federated learning. Background Technology
[0002] With the rapid development of artificial intelligence technology, high-quality labeled data has become crucial for driving the improvement of model capabilities. However, the collection, sharing, and centralized processing of much data in the real world, especially data involving personal privacy, trade secrets, or protected by laws and regulations, face enormous challenges. For example, in the medical field, patient medical records and imaging data contain highly sensitive personal information; direct sharing and centralized labeling not only violate privacy regulations but also pose a risk of data breaches. In the financial sector, customer transaction records and credit information are similarly subject to strict protection.
[0003] Traditional supervised data annotation methods require raw data to be aggregated and labeled on a central server, which not only increases the risk of data leakage but may also violate relevant data sovereignty and privacy regulations. To mitigate these issues, techniques such as data augmentation, semi-supervised learning, active learning, and weakly supervised learning have emerged. Among them, weakly supervised learning uses imprecise, incomplete, or noisy labels to drive the model, effectively reducing the reliance on massive amounts of accurately labeled data. However, even weakly supervised annotation processes often require a certain degree of data sharing or interaction, which may still pose a risk of privacy leakage.
[0004] Federated learning, as a distributed machine learning paradigm, enables multiple data holders to collaboratively build a global model without directly sharing the original data. This decentralized collaborative approach effectively addresses data privacy and security issues. However, existing federated learning technologies primarily focus on directly updating model parameters using existing labels. How to combine federated learning with weakly supervised data generation and labeling to efficiently generate and label weakly supervised data suitable for downstream tasks while ensuring privacy remains a challenging technical problem to be solved. Summary of the Invention
[0005] To address the aforementioned technical issues, this application provides a privacy-preserving, weakly supervised data annotation system based on federated learning, comprising: a central server and a client module; The central server module is used to send the global model, receive local model parameters, generate an updated global model based on the local model parameters through federated aggregation and differential privacy, and send the global model to the client module. The global model is used to help client modules deploy local models; Several client modules are independently deployed on several data holders; The client module is used to receive the global model, deploy the local model according to the global model, complete the local data preprocessing, generate weak supervision signals, complete the local data labeling based on the weak supervision signals, adjust the local model parameters through the weak supervision meta-learning algorithm to obtain the local model parameters, and encrypt and transmit the local model parameters to the central server module. The local model is used to generate weak supervision signals and to complete local data labeling using the weak supervision signals. After the central server module sends the global model to the client module, the client module obtains local data, initializes the local model based on the global model, and preprocesses the local data. The client module generates a weak supervision signal through the local model and weakly supervises and labels the local data based on the weak supervision signal. The client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters and generate local model parameters. The client module encrypts the transmitted local model parameters and transmits them to the central server module. The central server receives the local model parameters and generates a new global model based on them.
[0006] Furthermore, the client module includes: a transmission unit, a data preprocessing unit, a weakly supervised generator, a data labeler, and a model local update unit; The transmission unit is used to receive the global model and encrypt and transmit the local model parameters to the central server module. The data preprocessing unit is used to preprocess local data; The weak supervision generator is used to generate weak supervision signals for local data; The data labeler is used to label data based on weak supervision signals; The model local update unit is used to adjust local model parameters and obtain local model parameters through a weakly supervised meta-learning algorithm.
[0007] Furthermore, the weak supervision signal includes: fuzzy label, label confidence, and local correlation.
[0008] Furthermore, the client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters, generating local model parameters by including the following steps: Construct the total loss function, optimize the local model based on the total loss function, and obtain the local model parameters; The local model parameters are preprocessed, the local maximum influence parameter is calculated, and the local model parameters are generated based on the local model parameters and the local maximum influence parameter.
[0009] Furthermore, the steps of constructing the total loss function, optimizing the local model based on the total loss function, and obtaining the local model parameters include the following: Soft targets are generated through a local model, local prediction heads are generated based on the soft targets, and knowledge distillation loss is constructed based on the soft targets and local prediction heads. Weakly perturbed data is generated from local data, local features are generated from local data, weakly perturbed features and weakly perturbed supervision signals are generated from the weakly perturbed data, and a consistency loss is constructed from the local features, weakly perturbed supervision signals, weakly perturbed features and weakly perturbed supervision signals. Construct an alignment loss based on local features and weak supervision signals; A total loss function is constructed by combining knowledge distillation loss, consistency loss, and alignment loss. The local model is then optimized using the total loss function to obtain local model parameters.
[0010] Furthermore, the local maximum influence parameters include: clipping threshold and norm; The preprocessing of local model parameters, calculation of local maximum influence parameters, and generation of local model parameters based on local model parameters and local maximum influence parameters include the following steps: Calculate local updates based on local model parameters and obtain the update vector; Calculate the norm based on the updated vector; Obtain the clipping threshold, scale the update vector based on the clipping threshold and norm, and obtain the local model parameters.
[0011] Furthermore, the central server module generates a new global model based on local model parameters, including the following steps: Sum the local model parameters to obtain the aggregate parameters; Gaussian noise is injected into the aggregation parameters to obtain the noise aggregation parameters; Calculate the average value of the noise aggregation parameters to obtain the global update amount; The global model is updated based on the global update amount, and a new global model is generated.
[0012] The beneficial effects of this invention are as follows: 1. This invention constructs a complete, end-to-end privacy-preserving weakly supervised data generation and annotation closed loop. The generation and initial annotation of weakly supervised signals are both completed locally on the client side, ensuring the highest level of data security. 2. Break down data silos, enable multi-node collaboration, and build models without sharing data, thereby allowing the generation rules of weakly supervised signals to be collaboratively optimized globally; 3. Through the federated learning framework, the global model can be continuously improved, and the quality of the weak supervision signals and annotations generated locally by the client can also be indirectly improved, forming a virtuous cycle. The prior knowledge provided by the global model can guide the client to generate weak supervision signals more accurately. 4. This invention is not limited to a specific form of weak supervision signal, but supports the collaborative optimization of multiple forms of weak supervision signals under the federated learning framework; 5. This invention has strong versatility and can be applied to various data types and various weakly supervised generation strategies. By replacing different weakly supervised generators and aggregation algorithms, it can adapt to different application scenarios. 6. It overcomes the centralized bottleneck of traditional data annotation by distributing annotation tasks to various data sources, while achieving centralized extraction at the model level through federated learning, thus balancing efficiency and privacy. Attached Figure Description
[0013] Figure 1 This is a structural diagram of the privacy-protected weakly supervised data annotation system provided in an embodiment of the present invention. Detailed Implementation
[0014] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings.
[0015] like Figure 1 As shown, the present invention provides a privacy-preserving weakly supervised data annotation system based on federated learning, comprising: a central server and a client module; The central server module is used to send the global model, receive local model parameters, generate an updated global model based on the local model parameters through federated aggregation and differential privacy, and send the global model to the client module. The global model is used to help client modules deploy local models; Several client modules are independently deployed on several data holders; The client module is used to receive the global model, deploy the local model according to the global model, complete the local data preprocessing, generate weak supervision signals, complete the local data labeling based on the weak supervision signals, adjust the local model parameters through the weak supervision meta-learning algorithm to obtain the local model parameters, and encrypt and transmit the local model parameters to the central server module. The client module includes: a transmission unit, a data preprocessing unit, a weakly supervised generator, a data labeler, and a model local update unit; The transmission unit is used to receive the global model and encrypt and transmit the local model parameters to the central server module. The data preprocessing unit is used to preprocess local data; The preprocessing of local data includes: image normalization and text segmentation; The weak supervision generator is used to generate weak supervision signals for local data; The data labeler is used to label data based on weak supervision signals; Based on the generated weak supervision signal, the client assigns an initial weak supervision label to the data locally; for example, if the weak supervision signal indicates that the image is likely to contain a cat, then the image is assigned a vague label of "cat", possibly accompanied by a confidence score.
[0016] The model local update unit is used to adjust local model parameters and obtain local model parameters through a weakly supervised meta-learning algorithm.
[0017] The client module uses its local data and assigned weakly supervised labels to adjust the local model parameters using a weakly supervised meta-learner tuning algorithm. This enables the local model to better understand and utilize weakly supervised signals and gradually improve its ability to understand the data.
[0018] The local model is used to generate weak supervision signals and to complete local data labeling using the weak supervision signals. The weak supervision signals include: fuzzy labels, label confidence, and local correlation.
[0019] Generating weakly supervised signals specifically refers to generating weakly supervised signals for local data using pre-defined rules or a pre-built auxiliary model that has already completed preliminary learning on other public datasets; the pre-defined rules are, for example, based on keywords, image segmentation masks, template matching, etc.; the auxiliary model is, for example, a simple classifier or object detector.
[0020] Specifically, fuzzy labels include, for example, “This photo may contain a cat” or “This text may express a positive emotion”; label confidence assigns a confidence score between 0 and 1 to the data, indicating the reliability of the label; and local relevance marks areas in an image that may contain objects of interest, or highlights keywords in text.
[0021] After the central server module sends the global model to the client module, the client module obtains local data, initializes the local model based on the global model, and preprocesses the local data. The client module generates a weak supervision signal through the local model and weakly supervises and labels the local data based on the weak supervision signal. The client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters and generate local model parameters. The client module encrypts the transmitted local model parameters and transmits them to the central server module. The central server receives the local model parameters and generates a new global model based on them.
[0022] The client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters, and the generation of local model parameters includes the following steps: Construct the total loss function, optimize the local model based on the total loss function, and obtain the local model parameters; The steps of constructing the total loss function, optimizing the local model based on the total loss function, and obtaining the local model parameters include: Soft targets are generated through a local model, local prediction heads are generated based on the soft targets, and knowledge distillation loss is constructed based on the soft targets and local prediction heads. Knowledge distillation loss utilizes the aggregated knowledge of the global model to generate more reliable soft targets for local data, guiding the local model's learning.
[0023] Specifically, reducing local raw data Input to the local model of the current client To obtain a prediction under a global view ,These As Soft targets capture the class probability distribution information of the global model.
[0024] Then, a small, lightweight local auxiliary model, such as a shallow MLP or linear classifier, is used, called the local prediction head. ,this Learn from Features predict .
[0025] This leads to the construction of knowledge distillation loss. ;in, This is the Kullback-Leibler divergence, used to measure the similarity between two probability distributions; the goal of this step is to make... Learning Extraction and Features that are consistent with predictions.
[0026] Weakly perturbed data is generated from local data, local features are generated from local data, weakly perturbed features and weakly perturbed supervision signals are generated from the weakly perturbed data, and a consistency loss is constructed from the local features, weakly perturbed supervision signals, weakly perturbed features and weakly perturbed supervision signals. Consistency loss ensures that the local model has internal consistency when processing local data and its weak supervision signals, even in the face of noise or data perturbation.
[0027] Specifically, for each piece of local data Generate a weakly perturbated version For example, random cropping, flipping, and color jittering of images; synonym replacement and word order shuffling of text; etc. and All input to the local model Feature extractor , to obtain features and ;Will and the corresponding initial weak supervision signal Input to the weakly supervised parser The refined weak supervision representation and Finally, in order to measure and Consistency between them, thus constructing consistency loss ;in, It can be mean squared error, cosine similarity, or cross-entropy. Consistency loss encourages the local model to learn features that are robust to local perturbations and weakly supervised parsing capabilities.
[0028] Construct an alignment loss based on local features and weak supervision signals; Alignment loss ensures that the features learned by the local model and the parsing ability of weak supervision can effectively utilize the weak supervision signals. .
[0029] Specifically, direct comparison Output and itself; alignment loss: For example, if It is a confidence level Fuzzy labels Therefore, a weighted cross-entropy can be used: This prompts the model to respect the original weak supervision signals with high confidence.
[0030] A total loss function is constructed by combining knowledge distillation loss, consistency loss, and alignment loss. The local model is then optimized using the total loss function to obtain local model parameters.
[0031] Finally, the total loss function is constructed by weighted summing of the above losses: in These are hyperparameters used to balance the importance of various losses.
[0032] Then, update the local model using standard optimizers such as Adam or SGD, but only for a small number of iterations, e.g., 1-5 epochs, and with a learning rate much lower than typical training. The parameters, especially the feature extractor and weakly supervised parser Parameters; Local prediction head The parameters are usually temporary, or reset after each adjustment.
[0033] The key point is that this process is not for the purpose of... Instead of converging on a specific task, it fine-tunes or calibrates its internal representation and understanding of weakly supervised signals through multifaceted constraints, aligning it with global knowledge and making it more robust to local weakly supervised signals.
[0034] The local model parameters are preprocessed, the local maximum influence parameter is calculated, and the local model parameters are generated based on the local model parameters and the local maximum influence parameter.
[0035] Limiting the maximum impact of each client on the global model update before transmitting local model parameter updates to the central server is a key prerequisite for calculating the scale of differential privacy noise, as it limits the sensitivity.
[0036] The local maximum influence parameters include: clipping threshold and norm; The preprocessing of local model parameters, calculation of local maximum influence parameters, and generation of local model parameters based on local model parameters and local maximum influence parameters include the following steps: Calculate local updates based on local model parameters and obtain the update vector; Specifically, client After completing the local weakly supervised data generation, annotation, and local model update, the local model is obtained. Assume the global model is Then the client's update vector can be represented as .
[0037] Calculate the norm based on the updated vector; Calculate this update vector L2 norm: .
[0038] Obtain the clipping threshold, scale the update vector based on the clipping threshold and norm, and obtain the local model parameters.
[0039] Set a globally shared pruning threshold. C is a hyperparameter, typically determined by the central server during initialization; if Then Scaling is performed so that its norm equals ; The expression is: In this way, each client sends an update. The L2 norm will be less than or equal to .
[0040] This invention emphasizes that the pruning object is the adjustment amount of local model parameters rather than gradients in the traditional sense, and this pruning is performed by the client after generating local parameter updates, which is a limitation on its contribution and directly serves the differential privacy protection in subsequent federated aggregation.
[0041] By using the weakly supervised meta-learning algorithm, the client can not only use the wisdom of the global model to make up for the deficiencies of the local weak supervision signal, but also improve its own feature extraction ability on local data and its ability to analyze weak supervision signals. Thus, without leaking the original data, it lays the foundation for the iterative optimization of the global model and the final generation of high-quality weakly supervised data.
[0042] The central server module generates a new global model based on local model parameters, including the following steps: Sum the local model parameters to obtain the aggregate parameters; The central server from those participating in this round of aggregation Each client collects updates to the local model parameters after cropping. The aggregation parameters are obtained by summing. .
[0043] Gaussian noise is injected into the aggregation parameters to obtain the noise aggregation parameters; To aggregation parameters Gaussian noise is injected into the data. Gaussian noise is widely used in differential privacy due to its good combinatorial properties and applicability to multidimensional data.
[0044] in, It is a with A random vector of the same dimension, each element of which is independently selected from a vector with a mean of 0 and a standard deviation of 0. Gaussian distribution Sampling.
[0045] Among them, standard deviation The calculation is crucial. The size determines the strength of privacy protection; it is usually based on the privacy budget. , clipping threshold and the number of participating clients To calculate; a common calculation formula is: in, It represents the proportion of clients actually participating in the aggregation; more accurate calculations require considering the number of iterations and combinatorial characteristics of federated learning, and using privacy accounting techniques to track accumulated privacy losses.
[0046] Calculate the average value of the noise aggregation parameters to obtain the global update amount; Dividing the sum with noise by the number of participating clients yields the global update amount with differential privacy protection. .
[0047] The global model is updated based on the global update amount, and a new global model is generated.
[0048] Update the global model: The central server uses this privacy-preserving global update volume to update the global model. The expression is: .
[0049] Each iteration of the differential privacy mechanism consumes a portion of the privacy budget. To provide meaningful privacy guarantees throughout the federated learning process, the central server needs to use a privacy accounting mechanism to track the accumulated privacy costs. and Value; the training process should stop when the accumulated privacy budget reaches a preset limit.
[0050] In some cases, it is possible to design or The decay strategy allows for greater privacy loss in the early stages of training to accelerate convergence, while protecting privacy more strictly in the later stages.
[0051] This invention emphasizes that noise is injected into the aggregation parameters, rather than into each client's update, which aligns with the responsibilities of the central server; and it specifies the standard deviation of the Gaussian noise. Relying on privacy budgets, pruning thresholds, and the number of participating clients, this is a unique and practical detail for achieving differential privacy in a federated learning scenario.
[0052] The new global model is then transmitted from the central server module to each client module, thus completing the closed loop.
[0053] This invention provides robust privacy protection by introducing differential privacy into the aggregation process of federated learning. The core mechanisms include local update pruning on the client side to limit the sensitivity of individual contributions and noise injection on the central server side to mask individual contributions. Under the premise of ensuring mathematically provable privacy, this invention allows all parties to collaboratively build the model, effectively solving the privacy problem in sensitive data processing, which is the key to ensuring that this invention truly has privacy protection capabilities.
[0054] This invention combines the decentralized collaborative capabilities of federated learning with the flexibility of weakly supervised learning. The system can efficiently generate and label weakly supervised data while strictly protecting user data privacy. This invention solves key problems in traditional data labeling, such as privacy leakage, data silos, and dependence on high-cost, accurate labeled data, providing strong technical support for applications. Its core innovation lies in constructing a complete, privacy-friendly closed loop for weakly supervised data generation and labeling, and improving the quality of weakly supervised labeling through model iteration and optimization.
[0055] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.
Claims
1. A privacy-preserving, weakly supervised data annotation system based on federated learning, characterized in that, include: Central server and client modules; The central server module is used to send the global model, receive local model parameters, generate an updated global model based on the local model parameters through federated aggregation and differential privacy, and send the global model to the client module. The global model is used to help client modules deploy local models; Several client modules are independently deployed on several data holders; The client module is used to receive the global model, deploy the local model according to the global model, complete the local data preprocessing, generate weak supervision signals, complete the local data labeling based on the weak supervision signals, adjust the local model parameters through the weak supervision meta-learning algorithm to obtain the local model parameters, and encrypt and transmit the local model parameters to the central server module. The local model is used to generate weak supervision signals and to complete local data labeling using the weak supervision signals. After the central server module sends the global model to the client module, the client module obtains local data, initializes the local model according to the global model, and preprocesses the local data. The client module generates weakly supervised signals from the local model and weakly supervises the local data based on these signals. The client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters and generate local model parameters. The client module encrypts the transmitted local model parameters and transmits them to the central server module. The central server receives the local model parameters and generates a new global model based on them.
2. The privacy-preserving weakly supervised data annotation system according to claim 1, characterized in that, The client module includes: a transmission unit, a data preprocessing unit, a weakly supervised generator, a data labeler, and a model local update unit; The transmission unit is used to receive the global model and encrypt and transmit the local model parameters to the central server module. The data preprocessing unit is used to preprocess local data; The weak supervision generator is used to generate weak supervision signals for local data; The data labeler is used to label data based on weak supervision signals; The model local update unit is used to adjust local model parameters and obtain local model parameters through a weakly supervised meta-learning algorithm.
3. The privacy-preserving weakly supervised data annotation system according to claim 1, characterized in that, The weak supervision signals include: fuzzy labels, label confidence, and local correlation.
4. The privacy-preserving weakly supervised data annotation system according to claim 1, characterized in that, The client module uses a weakly supervised meta-learner tuning algorithm to adjust the local model parameters, and the generation of local model parameters includes the following steps: Construct the total loss function, optimize the local model based on the total loss function, and obtain the local model parameters; The local model parameters are preprocessed, the local maximum influence parameter is calculated, and the local model parameters are generated based on the local model parameters and the local maximum influence parameter.
5. The privacy-preserving weakly supervised data annotation system according to claim 4, characterized in that, The steps of constructing the total loss function, optimizing the local model based on the total loss function, and obtaining the local model parameters include: Soft targets are generated through a local model, local prediction heads are generated based on the soft targets, and knowledge distillation loss is constructed based on the soft targets and local prediction heads. Weakly perturbed data is generated from local data, local features are generated from local data, weakly perturbed features and weakly perturbed supervision signals are generated from the weakly perturbed data, and a consistency loss is constructed from the local features, weakly perturbed supervision signals, weakly perturbed features and weakly perturbed supervision signals. Construct an alignment loss based on local features and weak supervision signals; A total loss function is constructed by combining knowledge distillation loss, consistency loss, and alignment loss. The local model is then optimized using the total loss function to obtain local model parameters.
6. The privacy-preserving weakly supervised data annotation system according to claim 4, characterized in that, The local maximum influence parameters include: clipping threshold and norm; The preprocessing of local model parameters, calculation of local maximum influence parameters, and generation of local model parameters based on local model parameters and local maximum influence parameters include the following steps: Calculate local updates based on local model parameters and obtain the update vector; Calculate the norm based on the updated vector; Obtain the clipping threshold, scale the update vector based on the clipping threshold and norm, and obtain the local model parameters.
7. The privacy-preserving weakly supervised data annotation system according to claim 1, characterized in that, The central server module generates a new global model based on local model parameters, including the following steps: Sum the local model parameters to obtain the aggregate parameters; Gaussian noise is injected into the aggregation parameters to obtain the noise aggregation parameters; Calculate the average value of the noise aggregation parameters to obtain the global update amount; The global model is updated based on the global update amount, and a new global model is generated.