Gradient Noise Injection for Privacy in Federated Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for protecting sensitive data during machine learning model training, such as adding noise to gradient information, are not effective in preventing information leakage and can affect the accuracy of the training process.
Innovation Solution
Introduce noise only in the gradient change direction to generate an updated gradient, which increases the difficulty for attackers to obtain sensitive information while minimizing interference with the training process, ensuring the predictive model is trained along the correct loss gradient direction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If noise is added to gradient information to protect sensitive data, then sensitivity to attacks increases, but training accuracy deteriorates
Solution Approach 1:
The patent applies local quality by introducing noise only in specific directions (gradient change direction) rather than uniformly across all gradient components. This selective noise injection protects sensitive information in critical directions while preserving training accuracy in other directions, resolving the contradiction between attack resistance and training effectiveness.
Solution Approach 2:
The gradient update process is segmented into multiple components: the gradient change direction is identified and separated from other gradient components. Noise is then applied only to this specific segment, allowing differential protection strategies that maintain overall training accuracy while providing targeted sensitivity to attacks where it matters most.
2Reliability
If noise is added to prevent information leakage, then security improves, but resource overhead increases
Solution Approach 1:
Instead of applying noise uniformly to all gradient information (which would maximize security but also maximize resource overhead), the patent applies noise locally only to the gradient change direction. This reduces the amount of computation and resources required while maintaining essential security protections where they are most needed.
3Reliability
If noise is added in all directions, then information leakage is reduced, but interference with training process increases
Solution Approach 1:
The patent introduces noise only in the gradient change direction rather than in all directions. This localized approach provides protection against information leakage in the critical direction where sensitive information is most likely to be encoded, while minimizing interference with the training process in other directions where accuracy is maintained.
Solution Approach 2:
The gradient space is segmented into the gradient change direction and other directions. Noise is applied only to the segmented gradient change direction, creating a targeted protection mechanism that reduces overall interference with training while maintaining protection where it is most needed.
Data Source
AI summary
There are proposed a method, device, apparatus, and medium for protecting sensitive data. In a method, to-be-processed data is received from a server device. A processing result of a user for the to-be-processed data is received, the processing result comprising sensitive data of the user for the processing of the to-be-processed data. A gradient for training a server model at the server device is determined based on a comparison between the processing result and a prediction result for the to-be-processed data. The gradient is updated in a change direction associated with the gradient so as to generate an updated gradient to be sent to the server device. Noise is added only in the change direction associated with the gradient. The corresponding overhead of processing noise in a plurality of directions can be reduced, and no excessive noise data interfering with training will be introduced to the updated gradient.