Federated Semi-Supervised Learning for Sensitive Data Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning techniques for security classification of sensitive data face challenges due to data privacy concerns and geography-based regulations, leading to data silos and limitations in training data availability, especially in domains like text where data augmentation is not straightforward, and non-independent and identically distributed data across users poses significant challenges in federated learning.
Innovation Solution
The implementation of a data augmentation-free federated semi-supervised learning approach using model contrastive and distillation learning, which learns generalized representations without requiring data augmentation, and employs a client-server architecture with local and global models to classify sensitive data, adapting to different domains and handling non-IID data distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional deep learning techniques are used for security classification, then classification accuracy can be improved with large amounts of data, but data privacy concerns and geography-based regulations prevent data from being centralized, leading to data silos
Solution Approach 1:
The system segments the training process into local model training at client devices and global model aggregation at the server. Each client trains a local model on its own private data without sharing the data itself, while the server aggregates these local models to create a global model. This segmentation allows classification accuracy to improve through multiple clients' contributions while maintaining data privacy and compliance with regulations.
Solution Approach 2:
The server acts as an intermediary that coordinates federated learning across multiple clients. It distributes the global model to clients, collects local model updates, and aggregates them into an improved global model. This intermediary mechanism enables collaborative learning without direct data sharing between clients, resolving the contradiction between accuracy improvement and data sharing restrictions.
2Reliability
If data is kept in silos at end-user devices to preserve privacy, then data security is improved, but training data availability for deep learning techniques deteriorates
Solution Approach 1:
The system merges the training capabilities of multiple local models into a single global model through federated averaging. While data remains distributed and secure at each client device, the model parameters are combined at the server to create a global model that has learned from all clients' data. This merging achieves the benefits of large-scale training data availability while maintaining data security through cryptographic and architectural safeguards.
3Measurement precision
If data augmentation is used to increase training data, then model performance can be improved, but in text domains data augmentation is not straightforward and may compromise data integrity
Solution Approach 1:
The system uses self-service through semi-supervised learning where the model learns from both labeled and unlabeled data. The unlabeled data provides additional training signals without requiring manual augmentation or annotation. The model automatically learns from the structure and patterns in unlabeled text data, improving performance without the complexity and potential integrity issues of traditional data augmentation methods.
4Reliability
If federated learning is used to preserve privacy, then data privacy is maintained, but non-independent and identically distributed data across users poses significant challenges
Solution Approach 1:
The system employs semi-supervised learning with contrastive loss functions that change the parameter optimization approach. By using contrastive loss on unlabeled data alongside supervised loss on labeled data, the system adapts to non-IID distributions across clients. This parameter change in the loss function allows the model to learn robust representations even when data distributions vary significantly across different user devices, maintaining privacy while handling data heterogeneity.
Data Source
AI summary
This disclosure relates generally to system and method for classification of sensitive date using federated semi-supervised learning. Federated learning has emerged as a privacy-preserving technique to learn one or more machine learning (ML) models without requiring users to share their data. In federated learning, data distribution among clients is imbalanced resulting with limited data in some clients. The method includes extracting a training dataset from one or more data sources and pre-processing the training dataset into a machine readable form based on associated data type. Further, a federated semi-supervised learning model is iteratively trained based on a model contrastive and distillation learning to classify sensitive data from the unlabeled dataset. Then, sensitive data from a user query is received as input which are classified using the federated semi-supervised learning model.