Trustworthy and decentralized aggregation in federated learning

Abstract

The technique for distributed federated learning utilizes a multi-layered defense strategy to reduce information leakage. Instead of centrally aggregating model updates, aggregation is distributed across multiple independent and functionally equivalent execution entities, each running within its own Trusted Execution Environment (TEE). The TEE enables secret and remotely authenticated federated aggregation. Preferably, each aggregator entity runs within an encrypted virtual machine supporting in-memory encryption. Each party remotely authenticates its TEE before participating in training. By using multiple decentralized aggregators, parties can partition their respective model updates at the model parameter granularity and map individual weights to specific aggregator entities. Parties can also dynamically shuffle fragmented model updates at each training iteration to further obfuscate the information assigned to each aggregator execution entity. This architecture prevents single points of failure among aggregators and protects the model even if all aggregators are compromised.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to techniques for federated machine learning. BACKGROUND

[0002] Federated learning (FL) provides a collaborative training mechanism that allows multiple parties to jointly build a machine learning (ML) model. Rather than centralizing all training data in a central training server (or data center), FL allows parties to keep private data in their trusted and protected domains / infrastructures. Parties train local models and only periodically upload model updates or gradients to a central aggregator server. The aggregator fuses the model updates and broadcasts the aggregated model back to the parties for model synchronization. The federated learning training setup has a unique advantage for preserving training data privacy. This is particularly attractive for mutually untrusted / competing training parties and holders of sensitive data (e.g., health and financial data) where sharing data is prohibited by law or regulation.

[0003] There is a misconception in federated learning that the model updates exchanged in the FL communication contain less information than the original training data. It is concluded from this that sharing model updates is considered “privacy-preserving”. However, model updates are directly derived from the local training data. Although it can not be explicitly identifiable, the training data information is still hidden in the representation of the model updates. Recent research has challenged the privacy promise of federated learning. Specifically, this research shows that, assuming the central aggregator server is honest but curious, an adversary can perfectly infer privacy attributes or reconstruct the training data by exploiting the model updates.

[0004] Existing techniques to address these issues are by adding statistical noise to the model updates and using cryptographic primitives such as secure multi-party computation (SMC) protocols or homomorphic encryption (HE) for differential privacy aggregation. Both techniques have several drawbacks. The former typically significantly reduces the accuracy of the trained model and requires careful hyperparameter tuning, while the latter is computationally very expensive. Furthermore, because the parties do not trust each other, the central aggregator is often run on untrusted third-party (cloud) computing infrastructure and can become a single point of failure during an attack.

[0005] Therefore, there is still a need to provide an enhanced federated learning framework to address this threat model. SUMMARY

[0006] According to the present disclosure, federated learning systems and methods for neural network training to prevent privacy leakage and data reconstruction attacks are described. The methods herein provide enhanced protection against information leakage by exploiting a multi-layered defense strategy that includes multiple aspects, namely, trusted aggregation, decentralized aggregation with model partitioning, and dynamic permutation.

[0007] As used herein, trusted aggregation refers to the concept of using a trusted execution environment (TEE) that provides runtime memory encryption and remote attestation to facilitate isolated and confidential execution on untrusted servers. Decentralized aggregation refers to the concept of dividing a central aggregator into multiple independent and functionally equivalent execution entities, where each such entity then runs within an encrypted virtual machine. With multiple decentralized aggregators, parties can freely split model updates at model parameter granularity and map each weight to a specific aggregator. Thus, preferably, each aggregator has only a partial view of the model update and is unaware of the model architecture. By decentralizing a single aggregator with model update partitioning, this approach prevents the aggregator from becoming a single point of failure under a security attack, e.g., those against certain TEEs. Further, users can further deploy multiple aggregators to physical servers at different geographic locations and possibly with diverse TEEs on other microprocessors. Even if a subset of aggregators is compromised, an adversary cannot piece together the entire model update information.

[0008] In one exemplary implementation, each aggregator execution entity runs within an encrypted virtual machine (EVM) with runtime memory encryption. Before participating in training, each party of federated learning remotely authenticates that the hardware is genuine and establishes an end-to-end secure channel to exchange model updates.

[0009] According to an additional aspect, as described above, an additional defense strategy is referred to herein as dynamic permutation. Dynamic permutation takes advantage of the fact that arithmetic operations across model updates of federated learning fusion algorithms, e.g., federated stochastic gradient descent (FedSGD) and federated averaging (FedAvg), are bijective. Thus, partitioning and (internal) shuffling model updates does not affect the fusion result. According to this aspect of the disclosure, parties are provided with the ability to dynamically shuffle the sliced model updates at each training iteration to further obfuscate the information assigned to each aggregator execution entity. This strategy guarantees that even if all decentralized aggregators are compromised, an adversary cannot decrypt the correct ordering of model updates to reconstruct the training data. Dynamic permutation is enabled when the party-side transformation of model updates is deterministic, invertible, and identical across parties.

[0010] The foregoing has outlined some of the more pertinent features of the subject matter. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the subject matter as will be described. BRIEF DESCRIPTION OF DRAWINGS

[0011] For a more complete understanding of the subject matter and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 An exemplary block diagram of a distributed data processing environment in which aspects of the illustrative embodiments can be implemented is described;

[0013] Figure 2 An exemplary block diagram of a data processing system in which aspects of the illustrative embodiments can be implemented is described;

[0014] Figure 3 A cloud computing environment in which a fusion server implementing a secure distributed machine learning framework in accordance with the present disclosure is described;

[0015] FIG. 4 describes a distributed learning framework involving an aggregation server and a set of data owners / learning agents;

[0016] Figure 5 A first security technique referred to herein as trusted aggregation is described;

[0017] Figure 6 A system architecture implementing trusted and decentralized federated learning in accordance with the present disclosure is described; and

[0018] Figure 7 Representative implementations of model partitioning and dynamic replacement schemes are described. DETAILED DESCRIPTION

[0019] Referring now to the drawings and in particular to Figures 1-2 An exemplary diagram of a data processing environment in which illustrative embodiments of the present disclosure can be implemented is provided. It should be understood that Figures 1-2 are merely examples and are not intended to assert or imply any limitation on the scope of aspects or embodiments of the disclosed subject matter. Numerous modifications can be made to the depicted environments without departing from the spirit and scope of the present disclosure.

[0020] Referring now to the drawings, Figure 1 A pictorial representation of an exemplary distributed data processing system in which aspects of the illustrative embodiments can be implemented is depicted. The distributed data processing system 100 can include a network of computers in which aspects of the illustrative embodiments can be implemented. The distributed data processing system 100 contains at least one network 102, which is a medium that connects the different devices and computers together within the distributed data processing system 100 for providing communications links between them. The network 102 can include connections such as wire, wireless communication links, or fiber optic cables.

[0021] In the depicted example, server 104 and server 106 are connected to storage unit 108, which is also connected to network 102. In addition, clients 110, 112, and 114 are also connected to network 102. These clients 110, 112, and 114 can be, for example, personal computers, network computers, and the like. In the depicted example, server 104 provides data, such as boot files, operating system images, and applications to clients 110, 112, and 114. Clients 110, 112, and 114 are clients to server 104 in this example. Distributed data processing system 100 can include additional servers, clients, and other devices not shown.

[0022] In the depicted example, distributed data processing system 100 is the Internet with network 102 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, governmental, educational, and other computer systems that route data and messages. Of course, the distributed data processing system 100 can also be implemented to include a number of different types of networks, such as for example, an intranet, a local area network (LAN), a wide area network (WAN), or the like. As stated above, in Figure 1 are intended as examples only and are not intended to limit the scope of different embodiments of the disclosed subject matter. Figure 1 The particular elements illustrated in the should not be considered limiting for the environments in which the illustrative embodiments of the present application can be implemented.

[0023] Referring now to the Figure 2 , a block diagram of an exemplary data processing system which can implement aspects of the illustrative embodiments is shown. Data processing system 200 is an example of a computer, such as client 110 in Figure 1 in which computer usable code or instructions implementing the processes for illustrative embodiments of the disclosure can be located.

[0024] Referring now to the Figure 2 , a block diagram of a data processing system which can implement the illustrative embodiments is shown. Data processing system 200 is an example of a computer, such as server 104 or client 110 in Figure 1 in which computer usable program code or instructions implementing the processes for illustrative embodiments can be located. In this illustrative example, data processing system 200 includes communications fabric 202, which provides communications between processor unit 204, memory 206, persistent storage 208, communications unit 210, input / output (I / O) unit(s) 212, and display 214.

[0025] The processor unit 204 is configured to execute instructions for software that can be loaded into the memory 206. Processor unit 204 can be a set of one or more processors or can be a multi-processor core, depending on the particular implementation. Further, processor unit 204 can be implemented using one or more heterogeneous processor systems having main processors as well as secondary processors such as math co-processors or network co-processors. As another illustrative example, processor unit 204 can be a symmetric multi-processor (SMP) system including a plurality of processors of the same type.

[0026] Memory 206 and persistent storage 208 are examples of storage devices. A storage device is any hardware that is capable of storing information either on a temporary or permanent basis. In these examples, memory 206 can be, for example, a random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 208 can take various forms depending on the particular implementation. For example, persistent storage 208 can contain one or more components or devices. For example, persistent storage 208 can be a hard drive, a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. Media used by persistent storage 208 can also be removable. For example, a removable hard drive can be used for persistent storage 208.

[0027] In these examples, communication unit 210 provides communication through the use of either or both physical and wireless communications links. The communication unit 210 enables remote devices to communicate with one another.

[0028] Input / output unit 212 allows for input and output of data with other devices that can be connected to the data processing system 200. For example, input / output unit 212 can provide a connection for user input through a keyboard and mouse. Further, input / output unit 212 can send output to a printer. Display 214 provides a mechanism to display information to a user.

[0029] Instructions for the operating system and applications or programs are located on persistent storage 208. These instructions can be loaded into memory 206 for execution by processor unit 204. The processes of the different embodiments can be performed by the processor unit 204 using computer implemented instructions, which can be located in a memory, such as the memory 206. These instructions are referred to as program code, computer usable program code, or computer readable program code that can be read and executed by a processor in processor unit 204. The program code in the different embodiments can be embodied on different physical or tangible computer readable media, such as the memory 206 or the persistent storage 208.

[0030] The program code 216 is located in a functional form on a selectively removable computer readable media 218 and can be loaded or transferred into data processing system 200 for processing by processor unit 204. In these examples, program code 216 and computer readable media 218 form computer program product 220. In one example, computer readable media 218 can be in a tangible form, such as an optical or magnetic disk, inserted or placed into a drive or other device as part of permanent memory 208, for transfer onto a storage device, such as a hard disk drive as part of permanent memory 208. In a tangible form, computer readable media 218 can also take the form of permanent memory, such as a hard disk drive, thumb drive, or flash memory connected to data processing system 200. The tangible form of computer readable media 218 is also referred to as computer recordable storage media. In some instances, computer readable media 218 can not be removable.

[0031] Alternatively, program code 216 can be transferred to data processing system 200 from computer readable media 218 through a communications link to communications unit 210 and / or through a connection to input / output unit 212. In illustrative examples, the communications link and / or connection can be physical or wireless. Computer readable media can also take the form of non-tangible media, such as a communications link or wireless transmission containing program code. The different components shown for data processing system 200 are not meant to provide architectural limitations to the manner in which different embodiments can be implemented. The different illustrative embodiments can be implemented in a data processing system including components in addition to or in place of those illustrated for data processing system 200. Some components of data processing system 200 can be implemented as part of one or more programs, not necessarily implemented as part of an operating system. Figure 2 Other components shown can differ from the illustrative examples shown. As one example, a storage device in data processing system 200 is any hardware apparatus that can store data. Memory 206, permanent storage 208, and computer readable media 218 are examples of storage devices in a tangible form.

[0032] In another example, bus system can be used to implement communications fabric 202 and can include one or more buses, such as a system bus or an input / output bus. Of course, bus system can be implemented using any suitable type of architecture in which components can be connected by communicative coupling. Furthermore, communications unit can include one or more devices used to transmit and receive data, such as a modem or network adapter. Further, memory can be, for example, memory 206 or a cache such as found in an interface and memory controller hub that can be present in communications fabric 202.

[0033] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java TM , Smalltalk, C++ etc., as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0034] Those skilled in the art will appreciate that Figures 1-2 the hardware in Figures 1-2 may vary depending on the implementation. Other internal hardware or peripheral devices such as flash memory, equivalent non-volatile memory or optical disk drives and the like can be used in addition to or in place of the hardware depicted in Figures 1-2 . Also, the processes of the illustrative embodiments can be applied to a multiprocessor data processing system, other than the SMP system described earlier without departing from the spirit and scope of the disclosed subject matter.

[0035] As will be seen, the technology described herein can operate within a standard client-server paradigm, such as that shown in Figure 1 , where client machines communicate with an Internet-accessible network-based portal executing on a set of one or more machines. End users operate Internet-connectable devices (e.g., desktop computers, notebook computers, Internet-enabled mobile devices, etc.) that are capable of accessing and interacting with the portal. Typically, each client or server machine is a data processing system, such as that shown in Figure 2 , that includes hardware and software, and these entities communicate with one another over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. A data processing system typically includes one or more processors, an operating system, one or more applications, and one or more utilities. The applications on the data processing system provide native support for Web services including, but not limited to, support for HTTP, SOAP, XML, WSDL, UDDI, and WSFL, among others. Information regarding SOAP, WSDL, UDDI, and WSFL is available from the World Wide Web Consortium (W3C); further information about HTTP and XML is available from the Internet Engineering Task Force (IETF). Familiarity with these standards is assumed.

[0036] Applications on the data processing system can also use native support for non-standard protocols, or proprietary protocols developed to work over TCP / IP networks.

[0037] Cloud computing model

[0038] As noted above, the distributed machine learning techniques of the present disclosure preferably utilize computing elements located in a cloud computing environment. From this, the following additional background on cloud computing is provided.

[0039] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models, all generally described and defined by Peter Mell and Tim Grance in “The NIST Definition of Cloud Computing” published by the National Institute of Standards and Technology (NIST) in September 2011.

[0040] In particular, the following are illustrative characteristics:

[0041] On-demand self-service: cloud consumers can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically (without human interaction with the service’s provider).

[0042] Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0043] Resource pooling: the provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but can be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter).

[0044] Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly release to quickly scale in. To the consumer, the provisioned capacity can generally appear to be unlimited and can be purchased in any quantity at any time.

[0045] Measured Service: cloud systems automatically control and optimize resource use by leveraging utilization of capacity, often using meters that capture usage by individual account, as well as monitoring usage, controlling and reporting usage, providing transparency for both the provider and consumer of the service.

[0046] Service models are typically as follows:

[0047] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0048] Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0049] Infrastructure as a Service (IaaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include an operating system and / or application. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0050] Deployment models are typically as follows:

[0051] Private cloud: the cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0052] Community cloud: the cloud infrastructure is shared by several organizations and supports mission-oriented collaboration between said organizations. It can be managed by the organizations or a third party and can exist on-premises or off-premises.

[0053] Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

[0054] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0055] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is its infrastructure, which includes a network of interconnected nodes. Representative cloud computing nodes are shown above. Figure 2 As shown. Specifically, within a cloud computing node, there exists a computer system / server that can operate alongside many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with the computer system / server include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computers, and distributed cloud computing environments that include any of the above systems or devices. The computer system / server can be described in the general context of computer system executable instructions, such as program modules, executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system / server can be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0056] In a typical cloud computing environment, and as Figure 3 As shown, a set of functional abstraction layers is provided. These include hardware and software layers, virtualization layers, management layers, and workload layers.

[0057] The hardware and software layer 300 includes hardware and software components. Examples of hardware components include mainframes, in one example IBM® zSeries® systems; RISC (Reduced Instruction Set Computer) architecture based servers, in one example IBM pSeries® systems; IBM xSeries® systems; IBM BladeCenter® systems; storage devices; networks and networking components. Examples of software components include network application server software, in one example IBM WebSphere® application server software; and database software, in one example IBM DB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation in many jurisdictions

[0058] The virtualization layer 302 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers; virtual storage; virtual networks, including virtual private networks; virtual applications and operating systems; and virtual clients.

[0059] The management layer 304 provides various functions for the cloud computing environment. For example, resource provisioning provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal provides access to the cloud computing environment for consumers and system administrators. Service level management provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment provide pre-arrangement for, and procurement of, cloud computing resources for which future requirements are anticipated in accordance with an SLA.

[0060] The workloads layer 306 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation; software development and lifecycle management; virtual classroom education delivery; data analysis processing; transaction processing; enterprise specific functions in a private cloud; and distributed machine learning 308 in accordance with the present disclosure.

[0061] Accordingly, a representative cloud computing environment has a set of high-level functional components that include a front-end identity manager, a business support service (BSS) functional component, an operations support service (OSS) functional component, and a computing cloud component. The identity manager is responsible for interacting with requesting clients to provide identity management, and this component can be implemented with one or more known systems, such as the Tivoli Federated Identity Manager (TFIM) available from IBM Corporation of Armonk, New York. TFIM can be used to provide federated single sign-on (F-SSO) to other cloud components, as appropriate. The business support service component provides certain management functions, such as billing support. The operations support service component is used to provide provisioning and management of other cloud components, such as virtual machine (VM) instances. A virtual machine is an operating system or application environment installed on software but emulating a hardware machine. The cloud component represents the primary computing resources, which are typically a number of virtual machine instances for executing target applications that are available for access via the cloud. One or more databases are used to store directories, logs, and other working data. All of these components, including the front-end identity manager, are located "within" the cloud, but this is not required. In alternative embodiments, the identity manager can operate outside the cloud. The service provider can also operate outside the cloud.

[0062] Some clouds are based on non-traditional IP networks. Thus, for example, a cloud can be based on a dual-layer CLOS-based network with special single-layer IP routing using a hash of MAC addresses. The techniques described herein can be used in such non-traditional clouds.

[0063] Generally speaking, a cloud computing infrastructure provides a virtual machine hosting environment that includes hosts (e.g., servers or similar physical machine computing devices) and one or more management servers connected via a network. Typically, each physical server is adapted to dynamically provide one or more virtual machines using a virtualization technology such as VMware ESX / ESXi. Multiple VMs can be placed into a single host and share the host's CPU, memory, and other resources, increasing the utilization of an organization's data center. Among other tasks, the management server(s) monitor the infrastructure and automatically manipulate VM placement as needed, for example by moving virtual machines between hosts.

[0064] In non-limiting implementations, a representative platform technology is, but is not limited to, IBM System x® servers with VMware vSphere 4.1 Update 1 and 5.0.

[0065] Federated learning and threat model

[0066] A known method of distributed machine learning is depicted in FIG. 4. The system includes a fusion (aggregation) server 400, and N data owners or agents 402, sometimes referred to herein as learning agents. In this embodiment, each learning agent has access to a local dataset d i (typically consisting of labeled samples), and wants to train the same machine learning or neural network model. Each agent has a dataset that it wishes to protect, and cannot share with other agents or the aggregation server. In a typical operation, the distributed learning process can be performed as follows. At step (1), each agent 402 contacts the aggregation server 400 to obtain hyperparameters for training. In machine learning, a hyperparameter is a parameter whose value is set before the learning process begins; in contrast, other parameters’ values are derived via training. Each agent 402 trains the same type of neural network. In a representative example, the model associated with an agent is characterized by a parameter vector A = [pi... pk] that includes a plurality of parameters. There are multiple agents in the system, where A i is the parameter vector given by agent i. At step (2), the i-th agent trains the model on its local dataset d i , and this training is typically done by taking mini-batches, which are small subsets of the entire training data, and in doing so, the i-th agent computes its parameter vector A i . At step (3), each agent 402 sends the resulting parameters to the aggregation server 400, which then fuses (typically by computing the average or weighted average of the vectors) each parameter in the vectors. The average can use different priorities (weights m) for different agents; for example, if agent i gets weight m i , then the average parameter vector computed by the aggregation server is At step (4), the aggregation server 400 publishes the average parameter vector back to the agents 402. Steps (2)-(3) are then repeated for a given number of iterations until learning is deemed complete.

[0067] It turns out that the above process results in the same model that would be created if all the data were collected at a single location and used to train the model at least against an additive loss function (i.e., an indicator loss function (e.g., cross-entropy loss, norm-based loss, binary cross-entropy, etc.)). However, the challenge that this method faces is that it reveals the model to the aggregation server, which is typically a cloud-hosted service that the data owners / agents can not trust, while the data owners / agents themselves can not be located in the cloud. As will be described below, the multi-layer security techniques of the present disclosure solve this problem.

[0068] As further background, a key driver behind the emergence of federated learning was the need to address the privacy risks and constraints of centralized training, where training data must be collected from all parties and aggregated to a central server for training. In FL, training data remains decentralized on each participant's local device. Participants (or parties) must agree on the model architecture and maintain their local training pipelines. Instead of providing raw training data to a central server, as shown in Figure 4, each party uses its private training data to train its local model and uploads model updates to the central server. Typically, the aggregator is also responsible for managing the parties, orchestrating training tasks, and merging model updates. Aggregated model updates are distributed to the parties so that their local models are (usually) synchronized after each training iteration.

[0069] The following section provides further additional background on training DNNs in a federated learning distributed setup. Represented as model parameters and This is represented as a loss function. Each party has its own training data / label pairs. Each party can choose to share gradients for a data batch with the aggregator. The aggregator calculates the sum of gradients from all parties and enables each party to synchronize its model parameters locally. This fusion algorithm is called FedSGD. Alternatively, each party can train locally for several epochs and pass the model parameters: Upload to the aggregator. This aggregator can calculate the weighted average of the model parameters. ,in yes The size of the training data and It is all The aggregator then sends the aggregated model parameters back to the parties for synchronization. This model averaging fusion algorithm is called FedAvg. If only one batch of data is trained in a single FL training epoch and the model parameters are synchronized, FedAvg and FedSGD are equal because the gradient can be computed from the difference between the uploaded parameters of two consecutive models. Since FedAvg allows the parties to batch multiple SGD iterations before synchronizing updates, privacy attacks become more challenging because the model parameters become obfuscated with the injected data.

[0070] The FedSGD and FedAvg algorithms only involve bijective summation and averaging operations. Simply put, if the model is represented as an array, these fusion algorithms perform coordinate fusion between the parties. That is, they fused the model M1 from party P1 in an exponential manner. The parameters below are the same as those of the model M2 with the P2 square. The parameters are added or averaged under the given index - the differences between the parties. The parameters can be fused into any other exponent without knowledge of these parameters. This allows the entire model update to be divided into multiple segments, deployed to multiple servers, and the same fusion algorithm to be executed independently. Furthermore, parameters or gradients can be shuffled before aggregation, provided all parties perform the same permutations. For FL privacy attacks, the integrity of the model update and the data ordering are crucial for the optimization process of reconstructing the training data. The absence of either will lead to reconstruction failure. As will be described, the technique in this paper does not have such limitations, as it only requires that the parties can reverse the segmentation and permutations locally.

[0071] Federation learning can typically be used in both cross-device and cross-warehouse scenarios. Cross-device federated learning (FL) training usually involves a large number of mobile or Internet of Things (IoT) devices acting as clients. Clients are highly unreliable. These devices may frequently join and leave, and they are often power-limited due to their frequent reliance on batteries. However, cross-warehouse FL training usually involves a fixed number of organizations sharing a common motivation to collaboratively learn a model. They can provide reliable local training facilities. Therefore, the aggregator can maintain the state and address of each party with its unique ID. Cross-warehouse training places greater emphasis on data privacy, where strict data confidentiality is required. As will also be seen, the method presented in this paper addresses the problems in cross-warehouse FL training, but it is also applicable to cross-device domains.

[0072] The threat model in this paper assumes an honest but curious aggregation server. It is assumed that all parties involved in the FL training process are benign but do not tend to share training data with each other. An adversary attempts to inspect model updates uploaded from each party. The aim is to reconstruct the training data of all parties involved in FL training. This threat model is identical to FL privacy attacks. Furthermore, it is assumed that the parties involved in FL trust the System-on-Chip (SoC) hardware and the EVM that maintains the model aggregation workload.

[0073] System design

[0074] The following details a representative design of the federated learning framework of the present disclosure and describes how the method effectively mitigates the information leakage channels of FL privacy attacks. As noted above, the framework preferably utilizes a multi-layered security approach that includes (1) trusted aggregation, (2) decentralized aggregation, and (3) dynamic replacement. The first aggregation technique preferably implements a secret and trustworthy aggregation via a remote attestable encrypted virtual machine with runtime memory encryption (e.g., AMD® SEV EVM). The secure encrypted virtualization (SEV) used in the present example embodiment is a computing technology introduced by AMD in 2016. Its purpose is to protect security-sensitive workloads in public cloud environments. SEV relies on AMD secure memory encryption (SME) to implement runtime memory encryption. In conjunction with the AMD virtualization (AMD-V) architecture, SEV can enforce encrypted isolation between a guest VM and the hypervisor. Thus, SEV can prevent higher-privileged system administrators (e.g., at the hypervisor level) from accessing data within the domain of an encrypted virtual machine. When SEV is enabled, the SEV hardware tags all code and data of the VM with an address space identifier (ASID) that is associated with a unique temporary advanced encryption standard (AES) key known as the VM encryption key (VEK). The key is managed by the AMD SP, which is a 32-bit ARM Cortex-A5 microcontroller integrated in the AMD SoC. The runtime memory encryption is performed through on-chip memory controllers. Each memory controller has an AES engine that encrypts / decrypts data as it is written to main memory or read from the SoC. Control of memory page encryption is done through page tables. Physical address bit 47 (aka C-bit) is used to mark whether a memory page is encrypted or not. Similar to other TEEs, SEV also provides a remote attestation mechanism for authenticating the hardware platform and authenticating the guest VM to be launched. The authenticity of the platform is proven by an identity key signed by AMD and the platform owner. Before configuring any secrets, the guest VM owner verifies the authenticity of the SEV-enabled hardware and the measurement of the UEFI firmware, which helps to launch an encrypted virtual machine.

[0075] Figure 5 The general concept of secret aggregation is described. Assume that there are isolated and independent trusted execution environments in the cloud (one of which is shown at 500), such as described above with respect to Figure 3The described cloud execution environment. A TEE such as TEE 500 allows users to outsource their computations to a trusted third-party cloud server on the CPU package. Representative TEE technologies include, for example, Intel® SGX (Software Guard Extensions) / TDX (Trusted Domain Extensions), AMD® SEV (as described above), IBM® PEF (Protected Execution Facility), ARM TrustZone, and the like. TEEs are particularly attractive for collaborative ML computations because collaborative ML computations can involve large amounts of privacy-sensitive training data, multiple distributing parties, and more stringent data protection is the rule. Here, and as will be described, TEE 500 acts as a trusted intermediary for isolating the aggregator execution entity from other such execution entities that facilitate federated learning.

[0076] As Figure 5 shown, TEE 500 is associated with an operating system-based container mechanism (e.g., an open-source container such as a Kata container) for packaging and deployment, and whose execution is isolated in a virtual machine. Specifically, TEE 500 runs aggregator 502 within an encrypted virtual machine (EVM) 504 supported by runtime memory encryption (such as SEV) 506. Executing the aggregator within a TEE can mitigate memory corruption attacks. Aggregator 502 is one of a set of decentralized aggregators that together make up the single aggregator depicted in FIG. 4. Each aggregator (such as aggregator 502) executes within an EVM 504, and the EVM memory is protected by a different ephemeral virtual machine encryption key (VEK). In this way, the confidentiality of the model aggregation computation is also protected from unauthorized users (e.g., system administrators) and privileged software running on the host server, such as a hypervisor or OS. As will be described, parties 508 to the federated learning authenticate the true SEV hardware / firmware remotely and establish an end-to-end secure channel for exchanging model updates prior to participating in training, as will be described. Specifically, remote attestation facilitated by attestation server 505 is used to provide hardware attestation of the aggregator and a load-time integrity check. As also depicted, each party 508 has data and its own machine learning (ML) infrastructure 510, and typically collaborates by exchanging attributes (e.g., model gradients).

[0077] The following provides a more detailed description of a representative deployment example for the federated learning framework of the present disclosure, below, as background.

[0078] A representative deployment example for the federated learning framework is shown in Figure 6The system is illustrated in FIG. 6. In this example, there are four (4) parties 600 (numbered 1 through 4) participating in federated learning, and the aggregation mechanism is split into three (3) aggregator execution entities 602 (numbered Aggregator 1 through 3). Each aggregator execution entity 602 executes within a TEE 604, and thus there are three TEEs (numbered TEE 1 through 3). Similar to traditional federated learning, in the approach herein, each party 600 needs to register with the aggregator execution entities 602 to participate in training. Each party needs to authenticate the TEE platform, e.g., via remote attestation, before registration. One aggregator execution entity initiates the training process by first notifying all parties. During the training phase, the aggregator participates in multiple training iterations with all parties. At each training iteration, the parties first synchronize the local model by downloading the latest model update from the aggregator, then produce a new model update using local training data, and upload it to the aggregator. The aggregator merges the model updates from all parties and distributes the aggregated version back to the parties. The global training ends once a predetermined training criterion is met, e.g., the FL training reaches a specified number of training iterations, or once the local model accuracy requirement is met, the parties can decide to exit the FL training. Unlike traditional FL, the deployment involves multiple aggregator execution entities 602 running within TEEs 604, instead of a single central aggregator as shown in FIG. 4. In this system, the aggregator execution entities 602 need to communicate with each other for training synchronization. In addition, an attestation server 606 is also deployed that is responsible for vouching for the workload of the aggregators and provisioning secrets, as will now be described.

[0079] Trusted aggregation

[0080] As mentioned previously, the model updates exchanged between the parties and the aggregator can contain essential information for reverse-engineering the private training data. The following techniques are used to eliminate the adversary’s channel to intercept and inspect the model updates in transit and also in use. In this design, preferably, cryptographic isolation for FL aggregation is enforced via mechanisms such as, but not limited to, SEV. As noted in Figure 5 , these aggregators execute within EVMs, and the memory of each EVM is protected with a different ephemeral VEK. In the embodiment shown in Figure 6 , establishing trust between the aggregator execution entities 602 and the parties 600 is divided into two phases:

[0081] Phase I: Launching a trusted aggregator

[0082] First, the SIM EVM is securely launched with a hypervisor running in it. To establish trust of the EVM, proofs are provided to authenticate (1) the platform is a real secure (e.g., AMD SEV-enabled) hardware that provides the required security properties, and (2) the Unified Extensible Firmware Image (UEFI) used to initiate the EVM is untampered. Once the remote attestation is completed, a secret is preferably provided to the EVM as a unique identifier of the trusted aggregator. The secret is injected into the EVM's encrypted physical memory and used for the aggregator attestation at Phase II described below. In Figure 6 In Step (1), the attestation server 606 is shown that facilitates the remote attestation. To this end, the EVM owner instructs the service provider (e.g., AMD® SP) to derive a certificate chain, e.g., from the platform Diffie-Hellman public key (PDH) down to the root (e.g., AMD root key (ARK)). The certificate chain can be verified by a root certificate. In addition, a digest of the UEFI image, SEV API version, and VM deployment policy are preferably also included in the attestation report along with the certificate chain.

[0083] The attestation report is sent to the attestation server 606, which is equipped with the root certificate to verify the certificate chain to authenticate the hardware platform. Thereafter, the attestation server 606 generates a boot blob and a guest owner Diffie-Hellman public key (GODH) certificate. These are sent back to the aggregation server 606 for negotiating a transport encryption key (TEK) and a transport integrity key (TIK) and launching the EVM through a Diffie-Hellman key exchange (DHKE). The UEFI measurement can be retrieved by the SP by pausing the EVM at launch time. This measurement is sent to the attestation server 606 to authenticate the integrity of the UEFI launch process. Only after this, the attestation server 606 generates an encapsulation secret, which preferably includes an ECDSA prime251v1 key. The hypervisor (not shown) injects this secret into the EVM's physical memory space as a unique identifier of the trusted aggregator and continues the launch process. The injection process for this secret preferably follows a remote attestation protocol, such as the first generation SEV remote attestation protocol. Other remote attestation protocols can be implemented, e.g., the upcoming SEV-SNP, further enhancing the integrity of the launch process.

[0084] Phase II: Aggregator attestation

[0085] The parties participating in the FL must ensure that they interact with trusted aggregators that have runtime memory encryption protection. To enable aggregator authentication, and as described above, in Phase I, the authentication server 606 configures the ECDSA private key as a secret during EVM deployment. This key is used to sign challenge requests and thus to identify legitimate aggregators. In Figure 6 In step (2), before participating in the FL, a party first authenticates the aggregators by participating in a challenge request protocol. To do so, the party 600 sends a randomly generated random number to the aggregator performing entity 602. The aggregator performing entity 602 digitally signs the random number using its corresponding ECDSA private key and then returns the signed random number to the requesting party. The party verifies whether the random number is signed using the corresponding ECDSA public key. If the verification is successful, the party 600 proceeds to register with the aggregator performing entity 602 to participate in the FL. Furthermore, preferably, a secure channel is provided to protect the communication between the aggregators and between the aggregators and the parties for updating the model parameters. The secure channel can be implemented using Transport Layer Security (TLS) to support mutual authentication between a party and the aggregator. In this way, all exchanged model updates are protected in use and transmission.

[0086] Decentralized aggregation with model partitioning

[0087] While enabling trusted aggregation provides significant advantages, it can not be sufficient on its own, as there is no guarantee that the TEE is all-powerful and that no security breach will be exposed in the future. Therefore, a second security layer of decentralized aggregation with model partitioning enhances the resilience of the system to ensure that even if the TEE is compromised due to a data leak, an adversary still cannot reconstruct the training data from the model updates. This aspect of the disclosure is now described in detail with respect to the representative implementation shown in Figure 6

[0088] As explained previously, each aggregator performing entity 602 runs within the EVM and is only responsible for a portion of the model updates. In Figure 6 In the example, three (3) aggregators are established and, as described above, each party authenticates and registers with all the aggregators. In this example, decentralized aggregation is implemented as follows.

[0089] Inter-aggregator training synchronization. ​A communication channel is maintained between the aggregators for training synchronization, e.g., step (3). Any one of the aggregators performing entity 602 can start a training iteration and, by default, become the initiator node. All other aggregators become follower nodes and wait for commands from the initiator. At each training iteration, the initiator first queries all parties to start local training and retrieve model updates for fusion. Thereafter, the initiator informs all follower nodes to pull their corresponding model updates, aggregate them together, and distribute the aggregated updates back to the parties.

[0090] Decentralized aggregation does increase the cost for illicitly obtaining model information at the aggregation point. The aggregators no longer hold model architecture information; they only see numerical vectors. Moreover, even missing a very small fraction of model updates can completely invalidate a data reconstruction attack. Thus, this protection scheme requires compromise of all TEE-protected aggregators to obtain a complete set of model updates.

[0091] While compromise of all TEE-protected aggregators is very difficult, the following description can be implemented to further protect federated learning from information leakage or other hazards, a third security layer, dynamic permutation.

[0092] Dynamic permutation

[0093] To this end, and to further obfuscate the information transmitted from the parties to the aggregators, a dynamic permutation scheme is preferably deployed to shuffle the partitioned model updates, preferably at each training iteration (or some other defined period). As described above, the dynamic permutation scheme is based on the insight that the order of parameters in the model updates is irrelevant to the fusion algorithm, while they are critical to the data reconstruction algorithm used in FL privacy attacks. With this data order obfuscated, even if they obtain the entire model updates, an adversary cannot generate reconstructed training data.

[0094] Random model partitioning .

[0095] Model partitioning and dynamic permutation are depicted in Figure 7 , and with respect to the three (3) aggregators depicted there. Specifically, the aggregators 702 (aggregators 1-3) correspond to the aggregators in Figure 6 . Prior to the start of training, an aggregator mapper 710 (data structure) is randomly generated for each DNN model to be trained. The parties select the proportion of model parameters for each aggregator, although this can be set to default. Moreover, the local parties must agree on the mapper 710, and thus the mapper 710 is shared by all parties participating in the FL training. In Figure 7 , the first party has a local model 712 under training. As Figure 7The k parameters of the local model 712 are mapped to three aggregators, i.e. aggregators {1-3} as shown, using the mapper 710, where the depicted shading and crosshatching represents the aggregator property of each parameter within the model. As also shown, the model updates are decomposed and rearranged for the different aggregators (steps (4) in Figure 6 to generate shuffled partitions. The shuffled partitions are then uploaded to the respective aggregators and fusion is performed to generate aggregated partitions. After the parties receive the aggregated model updates from the different aggregators, they unshuffles the aggregated model updates to the correct order. The same mapper 710 is then queried again to merge the model updates to the original positions within the local model (step (5) in Figure 6 In Figure 7 only one local model (trained and then merged) is depicted, but each party has its own such local model construction.

[0096] Preferably, the dynamic permutation scheme shuffles the partitioned model updates at each training iteration. Each permutation is seeded with a secret agreed upon by the parties (e.g. propagated through a trusted intermediary) and a dynamically generated training iteration ID. Thus, preferably, the permutation varies between each training iteration, but is identical across all parties. In other words, the method herein preferably dynamically shuffles the model updates with a deterministic permutation at each training iteration. The aggregators (i.e. the aggregation performing entities) merge the model updates, and the parties are responsible for restoring the order of the aggregated model updates. This method removes data leakage attacks by dynamically adjusting the order of uploaded model parameters. In this way, among other benefits, the method makes privacy protection of local training data more effective in federated learning.

[0097] Thus, according to this aspect, preferably the entire model update generated by a party is partitioned into multiple segments (partitions), where the partitions are deployed to multiple servers (aggregation performing entities) that independently execute the same fusion algorithm. Further, the parameters or gradients (or, more generally, elements) are also locally shuffled (i.e. permuted) before aggregation, as long as all parties perform the same permutation. It is only required that the parties can reverse the local side of the partitioning and permutation. The local model update partitioning and permutation can be performed periodically, e.g. at each training iteration or at some other defined period; in the alternative, the update partitioning and permutation occurs asynchronously.

[0098] The partitioning of the entire local model into multiple partitions and the permutation of one or more elements within each partition occurs at each modelling iteration. The same partitioning is applied by all partitions or not. Furthermore, the partitioning and / or permutation strategy can be applied to a centralized aggregator (the case where the number of aggregation entities is equal to 1); in such a case, there is no partitioning, but there is a permutation of the weights in the model update.

[0099] In general, the dynamic permutation scheme described above facilitates aggregated obfuscation in federated learning. A party to the federated learning (or more generally, a first system of a plurality of systems participating in the federated learning) determines that an update vector (or more generally, an update) should be transmitted for fusion. An obfuscation algorithm is then applied to obfuscate the update vector to generate an obfuscated update vector. A secret shared by each party can be used for this purpose, and each party to the federated learning uses the same shared secret to locally apply the obfuscation algorithm. As explained by the example in Figure 7 As explained by the example in

[0100] The techniques described herein provide significant advantages. As will be appreciated by those skilled in the art, the approach protects and shields aggregation in federated learning from reverse engineering attacks while maintaining low overhead and supporting many different deep learning models and frameworks. Moreover, the techniques herein provide multiple structured and randomized model partitioning mechanisms to break up exchanged model parameters. In this way, even if a subset of aggregators is compromised, an adversary still cannot reconstruct training data information. Further, the techniques herein enable learning participants to authenticate trusted hardware platforms and to authenticate workloads that are subject to federated learning, thus further ensuring that sensitive data is not exposed and transmitted without end-to-end cryptographic protection. Compared to traditional FL training, the techniques herein do not impact final model accuracy and convergence rate. At the same time, the approach significantly minimizes unnecessary party-to-aggregator information exposure, which is critical for conducting FL privacy attacks.

[0101] As described above, the approach herein leverages the unique arithmetic properties of federated learning fusion algorithms and provides architectural and protocol enhancements to mitigate potential information leakage channels. The described federated learning system preferably employs a three-layer security strategy, namely, secret trusted aggregation, decentralized model partitioning, and dynamic permutation of model updates. The federated learning system implementing these security strategies is immune to training data reconstruction attacks.

[0102] Further, while it is preferred that the three techniques be used together, this is not required. Thus, the federated learning framework implementing the techniques of the present disclosure can benefit from one or more of the following techniques and strategies. A first strategy enables a trusted and remotely authenticated model aggregation by leveraging secret computation techniques. A second strategy involves dispersing a single aggregator to at most multiple independent executing entities, preferably each executing entity has only a fragmented view of the model updates and is not concerned with the model architecture. A third strategy provides support for random and dynamic permutation of partitioned model updates at each training iteration to make data reconstruction algorithms infeasible. By implementing all three layers of security strategies, and as described above, the system is resistant to state-of-the-art federated learning privacy attacks and exhibits low performance overhead in practical deployments.

[0103] There are additional advantages. One is that the distributed learning approach does not require the generation of auxiliary inputs, and the training participants share only a subset of the obfuscated model parameters during the federated learning process. The trusted execution environment protects the confidentiality of the updated model in transit and aggregation. Further, the approach prevents a malicious or compromised aggregator from reconstructing the training data of the federated learning participants. The approach prevents both (i) honest but curious aggregators and (ii) malicious or compromised aggregators from reconstructing the private training data from the model updates. Another advantage is that the same approach can be used for different FL tasks and can achieve the same level of training performance as the baseline.

[0104] The above-described techniques can be implemented using any machine learning algorithm or computation that can be distributed in the manner described.

[0105] The subject matter can be implemented in whole or in part as a service. Generally, the trusted and decentralized aggregation for federated learning functionality can be provided as a standalone function, or it can leverage functionality from other ML-based products and services. For example, the security techniques herein can leverage known services and solutions, such as the IBM framework for federated learning (FFL), to support the described trusted aggregation, decentralized multi-aggregator with model partitioning, and dynamic model update permutation. Preferably, the aggregator application is containerized to facilitate its deployment, although this is not required. Kata containers can be employed to deploy the aggregator containers inside lightweight VMs. Thus, and as has been described, preferably each aggregator container runs in an SEV-protected EVM (or equivalent). To provide the TEE security functionality, and in this exemplary but non-limiting embodiment, an AMD EPYC 7642 (Rome) microprocessor running the firmware SEV API is used.

[0106] The above-described functionality can be implemented in whole or part as standalone methods, e.g., software-based functionality executed by a hardware processor, or it can be made available as a management service, including as a web service via a SOAP / XML interface. The specific hardware and software implementation details described herein are for illustrative purposes only and are not intended to limit the scope of the described subject matter.

[0107] More generally, computing devices in the context of the disclosed subject matter are all data processing systems which include hardware and software, such as Figure 2 as shown in FIG. 1, and these entities communicate with each other through a network such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. The applications on the data processing systems provide native support for Web-based and other known services and protocols, including but not limited to support for HTTP, FTP, SMTP, SOAP, XML, WSDL, UDDI, and WSFL, etc. Information regarding SOAP, WSDL, UDDI and WSFL is available from the World Wide Web Consortium (W3C); further information regarding HTTP, FTP, SMTP and XML is available from the Internet Engineering Task Force (IETF). Familiarity with these known standards and protocols is assumed.

[0108] The schemes described herein can be implemented in or in conjunction with different server-side architectures including simple n-tier architectures, secure systems, web portals, federated systems, etc. Also as noted, the technology herein can be practiced in a loosely-coupled server (including "cloud" based) environment, as described in connection with Figure 3 FIG. 1.

[0109] More generally, the subject matter described herein can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In preferred embodiments, the functionality is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc. Furthermore, as described above, the access control functionality based on identified context can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable medium can be any apparatus that can contain or store a program for use by or in connection with a computer or instruction execution system. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device). Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disc, and an optical disk. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read / write (CD-R / W) and DVD. The computer-readable medium is a tangible article which can be made and sold, to a specific design.

[0110] The computer program product can be a product having program instructions (or program code) to implement one or more of the described functions. Those instructions or code can be stored in a computer-readable storage medium in a data processing system after being downloaded by a remote data processing system via a network. Alternatively, these instructions or code can be stored in a computer-readable storage medium in a server data processing system and adapted to be downloaded to a remote data processing system for use within the remote system's computer-readable storage medium.

[0111] In representative embodiments, the fusion server and each agent are implemented in a special purpose computer, preferably in software executed by one or more processors. The software is maintained in one or more data stores or memories associated with the one or more processors, and the software can be implemented as one or more computer programs. Collectively, the special purpose hardware and software include the functionality described above.

[0112] While the above describes a particular order of operations performed by certain embodiments of the invention, it should be understood that such an order is exemplary and that alternative embodiments can perform the operations in a different order, combine certain operations, overlap certain operations, etc. References in the specification to a given embodiment indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic.

[0113] Finally, while given components of the system have been described separately, one of ordinary skill will appreciate that some functionality can be combined or shared in given instructions, program sequences, code portions, and the like.

[0114] The technology herein provides improvements over another technology or technical field (e.g., machine learning systems, security incident and event management (SIEM) systems, other security systems), as well as improvements over automated-based network security analysis.

[0115] Having described the subject matter, what is claimed is below.

Claims

1. A method of federated learning providing reduced model-aggregated information leakage, comprising: loading an aggregator execution entity included in a set of decentralized aggregator execution entities in a separate trusted execution environment acting as a trusted intermediary isolating each aggregator execution entity from other aggregator execution entities in the set, wherein the trusted execution environment utilizes different encryption keys to provide runtime memory encryption protection for the respective aggregator execution entities; registering each party in a plurality of parties with each aggregator execution entity included in the set; establishing and maintaining secure communication channels between the aggregator execution entities for training synchronization during federated learning; deploying portions of a partitioned model update of a machine learning model to each aggregator execution entity in the set; and launching execution of the set of decentralized aggregator execution entities to train the machine learning model using the portions of the partitioned model update, wherein the set of decentralized aggregator execution entities conduct a plurality of training iterations with the plurality of parties, wherein at each training iteration of the machine learning model, each party in the plurality of parties: obtains a latest model update from each aggregator execution entity in the set, generates a new model update using local training data, and provides the new model update to each aggregator execution entity in the set, wherein in response to receiving the new model update, the set communicates via the secure communication channels to merge the new model update into an aggregated machine learning model and distribute the aggregated machine learning model to each party in the plurality of parties.

2. The method of claim 1, further comprising authenticating integrity of the aggregator execution entities.

3. The method of claim 1, wherein, The plurality of parties decompose respective model updates at model parameter granularity and map individual weights to one of the set of decentralized aggregator execution entities.

4. The method of claim 1, wherein, An ordering of parameter or gradient elements in at least one partition is shuffled.

5. The method of claim 1, wherein, The aggregator execution entity executes one of: a different machine, a different data center, a different trusted execution environment architecture, and a different geographic location relative to at least one other aggregator execution entity in the set of decentralized aggregator execution entities.

6. The method of any one of claims 1-5, wherein, Loading the aggregator execution entity includes receiving a unique secret in the trusted execution environment and using the unique secret to authenticate the aggregator execution entity at registration of a party.

7. A device for federated learning providing reduced model-aggregated information leakage, comprising: a hardware processor; a computer memory holding computer program instructions executed by the hardware processor to provide federated learning providing reduced model-aggregated information leakage, the computer program instructions configured to: loading an aggregator execution entity included in a set of decentralized aggregator execution entities in a separate trusted execution environment that acts as a trusted intermediary that isolates each aggregator execution entity from other aggregator execution entities in the set, wherein the trusted execution environment utilizes different cryptographic keys to provide runtime memory encryption protection for the respective aggregator execution entities; registering each party in the multi-party with each aggregator execution entity included in the set; establishing and maintaining a secure communication channel between the aggregator execution entities for training synchronization during federated learning; deploying a portion of a partitioned model update of a machine learning model to each aggregator execution entity in the set; and launching execution of the set of decentralized aggregator execution entities to train the machine learning model using the portion of the partitioned model update, wherein the set of decentralized aggregator execution entities conduct a plurality of training iterations with the multi-party, wherein at each training iteration of the machine learning model, each party in the multi-party: obtains a latest model update from each aggregator execution entity in the set, generates a new model update using local training data, and provides the new model update to each aggregator execution entity in the set, wherein in response to receiving the new model update, the set communicates via the secure communication channel to merge the new model update into an aggregated machine learning model and distribute the aggregated machine learning model to each party in the multi-party.

8. The apparatus of claim 7, wherein, the computer program instructions are further configured to authenticate the integrity of the aggregator execution entities.

9. The apparatus of claim 7, wherein, the multi-party decomposes the respective model updates at a model parameter granularity and maps individual weights to one of the set of decentralized aggregator execution entities.

10. The apparatus of claim 7, wherein, an ordering of parameter or gradient elements in at least one partition is shuffled.

11. The apparatus of claim 7, wherein, the aggregator execution entity executes one of: a different machine, a different data center, a different trusted execution environment architecture, and a different geographic location relative to at least one other aggregator execution entity in the set of decentralized aggregator execution entities.

12. The apparatus of any one of claims 7 to 11, wherein, the computer program instructions configured to load the aggregator execution entities further include computer program instructions configured to receive a unique secret in the trusted execution environment and use the unique secret to authenticate the aggregator execution entities when registering a party.

13. A computer program product for use in a data processing system for providing federated learning with reduced model aggregation information leakage, the computer program product holding computer program instructions that when executed by the data processing system are configured to: loading an aggregator execution entity included in a set of decentralized aggregator execution entities in a separate trusted execution environment acting as a trusted intermediary isolating each aggregator execution entity from other aggregator execution entities in the set, wherein the trusted execution environment utilizes different encryption keys to provide runtime memory encryption protection for respective aggregator execution entities; registering each party in the multi-party with each aggregator execution entity included in the set; establishing and maintaining secure communication channels between aggregator execution entities for training synchronization during federated learning; deploying portions of partitioned model updates of a machine learning model to each aggregator execution entity in the set; and launching execution of the set of decentralized aggregator execution entities to train the machine learning model using the portions of partitioned model updates, wherein the set of decentralized aggregator execution entities conducts multiple training iterations with the multi-party, wherein at each training iteration of the machine learning model, each party in the multi-party: obtains latest model updates from each aggregator execution entity in the set, generates new model updates using local training data, and provides the new model updates to each aggregator execution entity in the set, wherein in response to receiving the new model updates, the set communicates via the secure communication channels to merge the new model updates into an aggregated machine learning model and distribute the aggregated machine learning model to each party in the multi-party.

14. The computer program product of claim 13, wherein, the computer program instructions are further configured to authenticate integrity of the aggregator execution entities.

15. The computer program product of claim 13, wherein, the multi-party decomposes respective model updates at model parameter granularity and maps individual weights to one of the set of decentralized aggregator execution entities.

16. The computer program product of claim 13, wherein, ordering of parameter or gradient elements in at least one partition is shuffled.

17. The computer program product of claim 13, wherein, the aggregator execution entity executes in one of: a different machine, a different data center, a different trusted execution environment architecture, and a different geographic location relative to at least one other aggregator execution entity in the set of decentralized aggregator execution entities.

18. The computer program product of any one of claims 13 to 17, wherein, the computer program instructions configured to load the aggregator execution entities further include computer program instructions configured to receive a unique secret in the trusted execution environment and use the unique secret to authenticate the aggregator execution entities when registering a party.

19. A federated learning system to prevent information leakage of model aggregation, comprising: a set of trusted execution environments; and an aggregator divided into a set of decentralized aggregator execution entities, wherein each aggregator execution entity is located in a given one of the set of trusted execution environments acting as a trusted intermediary isolating each aggregator execution entity from other aggregator execution entities in the set, wherein the trusted execution environment utilizes different encryption keys to provide runtime memory encryption protection for respective aggregator execution entities; wherein each party in the plurality of parties registers with each aggregator execution entity included in the set, establishes and maintains a secure communication channel between the aggregator execution entities for training synchronization during federated learning, deploys a portion of a partitioned model update of a machine learning model to each aggregator execution entity in the set, and wherein execution of the set of decentralized aggregator execution entities is initiated to train the machine learning model using the portion of the partitioned model update, wherein the set of decentralized aggregator execution entities performs a plurality of training iterations with the plurality of parties, wherein at each training iteration of the machine learning model, each party in the plurality of parties: obtains a latest model update from each aggregator execution entity in the set, generates a new model update using local training data, and provides the new model update to each aggregator execution entity in the set, wherein in response to receiving the new model update, the set communicates via the secure communication channel to merge the new model update into an aggregated machine learning model and distribute the aggregated machine learning model to each party in the plurality of parties.

20. The federated learning system of claim 19, wherein, The aggregator execution entity executes on one of: a different machine, a different data center, a different trusted execution environment architecture, and a different geographic location relative to at least one other aggregator execution entity in the set of decentralized aggregator execution entities.

21. The federated learning system of any one of claims 19-20, wherein, The plurality of parties decomposes the respective model updates at a model parameter granularity and maps individual weights to one of the set of decentralized aggregator execution entities.

Images