Secure Aggregation for Distributed Machine Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Secure multi-party computational techniques in distributed machine learning, such as secure aggregation, are effective in preserving privacy but face challenges in detecting malicious attacks like 'poison attacks' where some users intentionally skew the model, and existing solutions like blockchain increase computational costs due to ledger growth with many participants.

Innovation Solution

A method that compares aggregated characteristics of updates from subsets of nodes to a reference value to identify and isolate nodes contributing corrupt updates, preserving individual node privacy while maintaining model integrity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If secure aggregation is used to preserve privacy in distributed machine learning, then individual node privacy is protected, but the ability to detect malicious attacks is lost

Engineering Contradiction:

Improveprivacy of individual node updatesVSAvoiddetection of malicious attacks

Core Design Contradiction:

Loss of informationVSReliability

Solution Approach 1:

The system segments the set of computing nodes into multiple subsets and performs separate secure aggregation for each subset. This segmentation allows the system to compare results across subsets and identify anomalies caused by malicious nodes, thereby detecting attacks while still preserving the privacy of individual node updates through secure aggregation techniques.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system introduces an intermediary validation mechanism that compares aggregated characteristics from different subsets against reference values or each other. This intermediary comparison layer enables detection of malicious updates without requiring direct inspection of individual node data, thus maintaining privacy while improving reliability through cross-validation.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If blockchain is used to ensure accountability in federated learning, then immutability of input is achieved, but computational cost increases due to ledger growth

Engineering Contradiction:

Improveimmutability of input dataVSAvoidcomputational cost for verifying transactions

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system extracts only the essential accountability information (aggregated characteristics from subsets) and stores/computes with this condensed data rather than maintaining a complete blockchain ledger of all individual transactions. This extraction approach maintains the immutability and verification benefits of blockchain-like accountability while dramatically reducing computational costs by working with summarized data.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of using blockchain to store and verify every individual node's updates (which causes ledger growth), the system inverts the approach by using secure aggregation to condense updates first, then applying verification mechanisms to the aggregated results. This inversion reduces the data volume requiring verification while maintaining accountability.

Inventive Principle:

Principle #13The other way round (Inversion)

3Productivity

If all node updates are aggregated together in a single federation, then model training efficiency is maintained, but malicious nodes can successfully poison the model

Engineering Contradiction:

Improvemodel training efficiencyVSAvoidpoison attacks on model integrity

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The system segments nodes into multiple subsets for separate aggregation, allowing malicious updates in one subset to be detected through comparison with other subsets. This segmentation maintains training efficiency by processing subsets in parallel while providing defense against poison attacks through cross-subset validation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements feedback by comparing aggregated characteristics from different subsets and using this comparison to identify and isolate malicious nodes. The feedback mechanism allows the system to detect and respond to poison attacks in real-time during the training process, maintaining both efficiency and integrity.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20240256973A1Training a machine learning model using a distributed machine learning process

Publication Date: 2024.08.01 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

AI summary

There is provided a computer implemented method for use in a distributed machine learning process for training a machine learning model, wherein the training is distributed across a plurality of computing nodes and updates to the machine learning model, as determined by the plurality of computing nodes, are aggregated using secure multi party computation. The method includes: i) obtaining an aggregated characteristic of updates to the machine learning model provided by a first subset of the plurality of computing nodes; ii) comparing the aggregated characteristic to an equivalent reference; and iii) identifying whether the first subset of nodes are contributing updates that are corrupting the machine learning model, based on the comparison.