Federated Model Training With Cloud-Terminal Split for Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional model training methods face challenges in balancing data privacy and security, especially in large-scale online recommendation systems, where user privacy data cannot be centrally tracked and stored, and large models require significant computing power and storage, leading to high resource demands.
Innovation Solution
A federated learning approach is adopted, splitting the machine learning model into a cloud submodel and terminal submodels, allowing training across multiple terminals while ensuring privacy and reducing computational burden on individual devices, with the cloud submodel leveraging powerful servers for complex computations and terminal submodels handling simpler tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional model training methods are used, then model training can be performed, but data privacy cannot be protected and computing resources are consumed centrally
Solution Approach 1:
The patent divides the machine learning model into a cloud submodel and terminal submodels, with the cloud submodel containing parameters w1 and the terminal submodel containing parameters w2. This segmentation allows distributed training where the terminal performs local training with its own data while the cloud coordinates the overall process, thereby protecting data privacy while reducing central resource consumption.
2Reliability
If large models are used, then model effectiveness is improved, but computing power and storage requirements increase significantly
Solution Approach 1:
The patent segments the large model into cloud and terminal submodels, placing computationally intensive operations on the cloud while keeping the terminal submodel lightweight enough for edge devices. This allows the system to maintain model effectiveness without requiring excessive computing power and storage at the terminal level.
Solution Approach 2:
The cloud submodel acts as an intermediary that handles complex computations and coordinates training across multiple terminals. The cloud receives gradients from terminal submodels, aggregates them, and updates the cloud submodel parameters, thereby reducing the computational burden on individual terminals while maintaining overall model effectiveness.
3Ease of operation
If centralized data storage is used, then data can be accessed easily, but user privacy data cannot be protected
Solution Approach 1:
The patent segments data storage and processing between terminals and cloud. Each terminal stores and processes its own local data with the terminal submodel, while the cloud coordinates training without accessing individual user data. This segmentation maintains data accessibility for training while protecting user privacy.
Solution Approach 2:
Each terminal performs self-service training using its own local data and terminal submodel without requiring data to be sent to the cloud. The terminal independently trains its submodel using local data, then sends only model parameters and gradients to the cloud for coordination, thereby protecting privacy while maintaining training effectiveness.
4Productivity
If distributed training across multiple terminals is implemented, then training flexibility and speed are improved, but system complexity increases
Solution Approach 1:
The patent segments the training system into independent terminal submodels that can operate autonomously and a cloud coordinator that manages communication between terminals. This segmentation enables parallel training across multiple terminals, improving productivity while keeping individual terminal components simple and manageable.
Solution Approach 2:
The cloud submodel serves multiple functions: it coordinates training across terminals, aggregates gradients, updates model parameters, and manages communication between terminals. This multi-functionality at the cloud level simplifies the overall system architecture while enabling distributed training that improves productivity.
Data Source
AI summary
A model training method and apparatus, a system, and a storage medium. The model training method includes: obtaining a cloud training feature; training the cloud submodel by using the cloud training feature to obtain a cloud output result of the cloud submodel; sending the cloud output result and current parameters of the M terminal submodels to at least one terminal; receiving terminal gradients respectively output by N terminal submodels in the M terminal submodels that are output by the at least one terminal; calculating and obtaining a parameter gradient of the cloud submodel based on the terminal gradients respectively output by the N terminal submodels and the cloud output result; and adjusting current parameters of the N terminal submodels and a current parameter of the cloud submodel by using the parameter gradients of the N terminal submodels and the parameter gradient of the cloud submodel.