Method, apparatus, and computer program for federated learning with reduced information leakage (Federated learning with partitioned dynamic shuffle model updates)

Abstract

To provide a method, apparatus and computer program for federated learning with reduced information leakage.SOLUTION: In the method, instead of aggregating model updates centrally, an aggregation function is decentralized into multiple independent and functionally equivalent execution entities, each running within its own trusted executed environment (TEE). The TEEs enable confidential and remote-attestable federated aggregation. Each aggregator entity runs within an encrypted virtual machine that supports runtime in-memory encryption. Each party remotely authenticates the TEE before participating in the training. By using multiple decentralized aggregators, parties are enabled to partition their respective model updates at model-parameter granularity, and can map single weights to a specific aggregator entity. Parties also can dynamically shuffle fragmentary model updates at each training iteration to further obfuscate information dispatched to each aggregator execution entity.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates generally to techniques for distributed machine learning. [Background technology]

[0002] [Related Technology Background] Federated learning (FL) provides a collaborative training mechanism that allows multiple parties to collaboratively build machine learning (ML) models. Instead of pooling all training data in a central training server (or data center), federated learning allows each party to keep its private data within a trusted and protected domain / infrastructure. Each party trains a local model and only periodically uploads model updates or gradients to a central aggregation server. This aggregator fuses the model updates and broadcasts the aggregated model back to all parties for model synchronization. A federated learning training setup offers unique advantages in preserving the privacy of training data. It is particularly attractive for mutually distrustful / competing training parties and holders of sensitive data (e.g., health and financial data) that are prohibited from sharing data by law or regulation.

[0003] In federated learning, there has been a misconception that model updates exchanged in FL communication contain less information than the raw training data. For this reason, sharing model updates has been thought to be "privacy-preserving." However, model updates are derived directly from local training data. While they may not be explicitly identifiable, training data information is still hidden in the representation of model updates. Recent research has cast doubt on the privacy promise of federated learning. In particular, this research demonstrates that, assuming an honest-but-curious central aggregation server, an adversary can reliably use model updates to infer personal attributes or reconstruct training data. Summary of the Invention [Problem to be solved by the invention]

[0004] Existing techniques that address these issues include differentially private aggregation by adding statistical noise to model updates and using cryptographic primitives such as Secure Multi-Party Computation (SMC) protocols or Homomorphic Encryption (HE). Both techniques have several drawbacks: the former often significantly reduces the accuracy of the trained model and requires careful hyperparameter tuning, while the latter is computationally intensive. Also, because the parties do not trust each other, the central aggregator often runs on untrusted third-party (cloud) computing infrastructure, potentially becoming a single point of failure in the event of an attack.

[0005] Therefore, there remains a need to provide an enhanced federated learning framework that addresses this threat model. [Means for solving the problem]

[0006] According to the present disclosure, a federated learning system and method for neural network training to defend against privacy leaks and data reorganization attacks are described. The approach herein provides enhanced protection against information leaks by leveraging a defense-in-depth strategy comprising several aspects: trusted aggregation, distributed aggregation with model partitioning, and dynamic reordering.

[0007] As used herein, trusted aggregation refers to the concept of using a trusted execution environment (TEE) that provides runtime memory encryption and remote attestation to facilitate isolated confidential execution on an untrusted server. Distributed aggregation refers to the concept of splitting a central aggregator into multiple independent, functionally equivalent execution entities, and then running each such entity within an encrypted virtual machine. Multiple distributed aggregators allow parties the freedom to decompose model updates by model parameter and map each single weight to a specific aggregator. Therefore, each aggregator preferably has only a partial view of the model update and is unaware of the model architecture. By splitting a single aggregator by distributing model updates, this approach prevents the aggregator from becoming a single point of failure, for example, in a security attack targeting a specific TEE. Furthermore, users may deploy multiple aggregators on physical servers in different geographic locations and use diverse TEEs on other microprocessors. Even if one aggregator is compromised, an adversary cannot piece together updates to the entire model.

[0008] In one exemplary implementation, all aggregator execution entities run within an encrypted virtual machine (EVM) with runtime memory encryption. Before participating in training, each party in the federated learning remotely authenticates its hardware and establishes an end-to-end secure path for exchanging model updates.

[0009] According to an additional aspect, as mentioned above, an additional defensive strategy is referred to herein as dynamic reordering. Dynamic reordering leverages the notion that the arithmetic operations of federated learning fusion algorithms, such as Federated Stochastic Gradient Descent (FedSGD) and Federated Averaging (FedAvg), are dual across model updates. Thus, splitting and (internally) shuffling model updates does not affect the fusion results. According to this aspect of the present disclosure, parties are provided with the ability to dynamically shuffle the piecemeal model updates in each training iteration to further obfuscate the information dispatched to each aggregator execution entity. This strategy ensures that even if all distributed aggregators are compromised, an adversary cannot decipher the correct order of model updates to reconstruct the training data. Dynamic reordering is effective when party-side transformations of model updates are deterministic, reversible, and identical between parties.

[0010] The foregoing has outlined some of the more pertinent features of the present subject matter. These features should be construed as merely exemplary. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the subject matter as described. [Brief explanation of the drawings]

[0011] For a more complete understanding of the subject matter and its advantages, reference should now be made to the following description taken in conjunction with the accompanying drawings.

[0012] [Figure 1] FIG. 1 illustrates an exemplary block diagram of a distributed data processing environment in which exemplary aspects of the illustrative embodiments may be implemented. [Figure 2] 1 is an exemplary block diagram of a data processing system in which exemplary aspects of the illustrative embodiments may be implemented; [Figure 3]FIG. 1 illustrates a cloud computing environment in which a fusion server of a secure distributed machine learning framework according to the present disclosure may be implemented. [Figure 4] FIG. 1 illustrates a distributed learning framework that includes an aggregation server and a set of data owners / learning agents. [Figure 5] FIG. 1 illustrates a first secure technique, referred to herein as trusted aggregation. [Figure 6] FIG. 1 illustrates a system architecture for implementing the trusted distributed federated learning of the present disclosure. [Figure 7] FIG. 1 illustrates a representative implementation of a model splitting and dynamic sorting scheme. DETAILED DESCRIPTION OF THE INVENTION

[0013] Referring now to the drawings, and in particular to Figures 1-2, exemplary diagrams of data processing environments are provided in which exemplary embodiments of the present disclosure may be implemented. It should be understood that Figures 1-2 are exemplary only and are not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the disclosed subject matter may be implemented. Many modifications to the depicted environments may be made without departing from the spirit and scope of the present invention.

[0014] Referring now to the drawings, Figure 1 shows a pictorial representation of an exemplary distributed data processing system in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 may include a network of computers in which aspects of the exemplary embodiments may be implemented. Distributed data processing system 100 includes at least one network 102, which is the medium used to provide communications links between various devices and computers connected together within distributed data processing system 100. Network 102 may include connections such as wires, wireless communication links, or fiber optic cables.

[0015] In the illustrated example, servers 104 and 106 are connected to network 102 along with storage unit 108. Additionally, clients 110, 112, and 114 are also connected to network 102. Clients 110, 112, and 114 may be, for example, personal computers, network computers, etc. In the illustrated example, server 104 provides data to clients 110, 112, and 114, such as boot files, operating system images, applications, etc. In the illustrated example, clients 110, 112, and 114 are clients to server 104. Distributed data processing system 100 may include additional servers, clients, and other devices not shown.

[0016] In the depicted example, distributed data processing system 100 is the Internet, where network 102 represents a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes (host computers) made up of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, distributed data processing system 100 may be implemented to include multiple different types of networks, such as, for example, an intranet, a local area network (LAN), a wide area network (WAN), etc. As noted above, FIG. 1 is intended as an example, and not an architectural limitation for different embodiments of the disclosed subject matter, and thus the specific elements illustrated in FIG. 1 should not be considered limiting with regard to the environments in which illustrative embodiments of the present invention may be implemented.

[0017] 2, a block diagram of an exemplary data processing system 200 is shown in which aspects of the exemplary embodiments may be implemented. Data processing system 200 is an example of a computer, such as client 110 of FIG. 1, in which computer-usable code or instructions implementing the processes for the exemplary embodiments of the present disclosure may be located.

[0018] 2, a block diagram of a data processing system in which an illustrative embodiment may be implemented is shown. Data processing system 200 is an example of a computer, such as server 104 or client 110 in FIG. 1, in which computer-usable program code or instructions implementing processes in accordance with an illustrative embodiment may be located. In this illustrative example, data processing system 200 includes a communications fabric 202 that provides communications between a processor unit 204, a memory 206, persistent storage 208, a communications unit 210, an input / output (I / O) unit 212, and a display 214.

[0019] Processor unit 204 is responsible for executing instructions for software that may be loaded into memory 206. Processor unit 204 may be a set of one or more processors or may be a multi-processor core, depending on the particular implementation. Additionally, processor unit 204 may be implemented using one or more heterogeneous processor systems in which a main processor exists along with secondary processors on a single chip. As another illustrative example, processor unit 204 may be a symmetric multi-processor (SMP) system that includes multiple processors of the same type.

[0020] Memory 206 and persistent storage 208 are examples of storage devices. A storage device is any piece of hardware that can store information, either on a temporary basis, a persistent basis, or a combination thereof. In these examples, memory 206 may be, for example, random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 208 may take various forms depending on the particular implementation. For example, persistent storage 208 may include one or more components or devices. For example, persistent storage 208 may be a hard drive, flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The medium used for persistent storage 208 may be removable. For example, a removable hard drive may be used for persistent storage 208.

[0021] In these examples, communications unit 210 provides for communication with other data processing systems or devices. In these examples, communications unit 210 is a network interface card. Communications unit 210 may provide for communication using either or both physical and wireless communications links.

[0022] Input / output unit 212 may input and output data to and from other devices that may be connected to data processing system 200. For example, input / output unit 212 may provide a connection for user input by way of a keyboard and mouse. Additionally, input / output unit 212 may send output to a printer. Display 214 may provide a mechanism for displaying information to a user.

[0023] Instructions for the operating system and applications or programs are located in persistent storage 208. These instructions may be loaded into memory 206 for execution by processor unit 204. The processes of the different embodiments may be performed by processor unit 204 using computer-implemented instructions, which may be located in a memory, such as memory 206. These instructions are referred to as program code, computer usable program code, or computer readable program code, which may be read and executed by a processor in processor unit 204. The program code according to the different embodiments may be embodied in different physical or tangible computer-readable media, such as memory 206 or persistent storage 208.

[0024] Program code 216 is functionally disposed on computer-readable medium 218, which is selectively removable and can be loaded onto or transferred to data processing system 200 for execution by processor unit 204. In these examples, program code 216 and computer-readable medium 218 form computer program product 220. In one example, computer-readable medium 218 may be in a tangible form, such as an optical or magnetic disk that is inserted into or placed into a drive or other device that is part of persistent storage 208 for transfer onto a storage device, such as a hard drive that is part of persistent storage 208. In a tangible form, computer-readable medium 218 may also take the form of persistent storage, such as a hard drive, thumb drive, or flash memory that is connected to data processing system 200. The tangible form of computer-readable medium 218 is also referred to as a computer-recordable recording medium. In some examples, computer-recordable medium 218 may not be removable.

[0025] Alternatively, program code 216 may be transferred to data processing system 200 from computer-readable medium 218 over a communications link to communications unit 210, or a connection to input / output unit 212, or a combination thereof. The communications link, or connection, or a combination thereof, may be physical or wireless in the illustrative examples. The computer-readable medium may also take the form of an intangible medium, such as a communications link or wireless transmission containing the program code. The different components illustrated for data processing system 200 are not intended to provide architectural limitations to the manner in which different illustrative embodiments may be implemented. Different illustrative embodiments may be implemented in a data processing system including components in addition to or instead of those illustrated for data processing system 200. Other components illustrated in FIG. 2 may vary from the illustrated illustrative example. By way of example, storage devices in data processing system 200 may be any hardware device capable of storing data. Memory 206, persistent storage 208, and computer-readable medium 218 are examples of storage devices in a tangible form.

[0026] In another example, a bus system may be used to implement communications fabric 202 and may include one or more buses, such as a system bus or an input / output bus. Of course, the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between different components or devices coupled to the bus system. Additionally, a communications unit may include one or more devices used to transmit and receive data, such as a modem or a network adapter. Further, a memory may be, for example, memory 206 or a cache, such as found in an interface and memory controller hub that may be present in communications fabric 202.

[0027] Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider).

[0028] Those skilled in the art will appreciate that the hardware in Figures 1-2 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware depicted in Figures 1-2. Additionally, the processes of the illustrative embodiments may be applied to multiprocessor data processing systems other than the SMP systems described above without departing from the spirit and scope of the disclosed subject matter.

[0029] As can be seen, the techniques described herein may operate in conjunction within a standard client-server paradigm, such as that shown in FIG. 1, in which a client machine communicates with an Internet-accessible, web-based portal running on a set of one or more machines. End users operate Internet-enabled devices (e.g., desktop computers, notebook computers, Internet-enabled mobile devices, etc.) that can access and interact with the portal. Typically, each client or server machine is a data processing system, such as that shown in FIG. 2, including hardware and software, and these entities communicate with each other over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. A data processing system typically includes one or more processors, an operating system, one or more applications, and one or more utilities. Applications on the data processing system provide native support for Web services, including, but not limited to, support for HTTP, SOAP, XML, WSDL, UDDI, WSFL, etc. Information about SOAP, WSDL, UDDI, and WSFL is available from the World Wide Web Consortium (W3C), which is responsible for developing and maintaining these standards. Further information on HTTP and XML is available from the Internet Engineering Task Force (IETF). Familiarity with these standards is assumed.

[0030] Applications on a data processing system may use native support for non-standard protocols or private protocols developed to work over TCP / IP networks.

[0031] [Cloud Computing Model] As mentioned above, the distributed machine learning techniques of the present disclosure preferably utilize computing elements located in a cloud computing environment, and therefore, additional background regarding cloud computing is provided below.

[0032] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models, all of which are described and defined in more detail in "The NIST Definition of Cloud Computing" by Peter Mell and Tim Grance (September 2011).

[0033] Specifically, the following are some of the representative features:

[0034] On-Demand Self-Service: Cloud consumers can unilaterally set up computing capabilities such as server time and network storage automatically when needed, without the need for human interaction by the provider of the service.

[0035] Broad network access capabilities are available across the network and access by standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (eg, cell phones, laptops, and PDAs).

[0036] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model with different physical and virtual resources dynamically allocated and reallocated according to demand. There is an implication of location independence in that consumers generally do not have control or knowledge of the exact location of the resources provided, but may be able to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0037] Rapid Elasticity: Capabilities can be rapidly and elastically provided, in some cases automatically, to quickly scale out, and rapidly released to quickly scale in. To the consumer, each capacity available for provisioning often appears unlimited, and they can purchase as much as they need, at any time.

[0038] Measured Services: Cloud systems automatically control and optimize resource usage by leveraging measurement capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of utilized services.

[0039] The service model is typically as follows:

[0040] Software as a Service (SaaS): The consumer is offered the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). With the expected exception of limited user-specific application configuration settings, the consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions.

[0041] Platform as a Service (PaaS): The ability offered to consumers is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration.

[0042] Infrastructure as a Service (IaaS): The capability offered to consumers is the provision of processing, storage, network, and other underlying computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating system, storage, deployed applications, and in some cases, limited control of selected networking components (e.g., host firewalls).

[0043] The deployment model is typically as follows:

[0044] Private Cloud: The cloud infrastructure operates solely for an organization. A private cloud may be managed by the organization or a third party and may reside on-premise or off-premise.

[0045] Community Cloud: Cloud infrastructure is shared by several organizations and supports a specific community with shared relationships (e.g., mission, safety requirements, policies, and compliance considerations). The community cloud may be managed by those organizations or a third party and may exist on-premises or off-premises.

[0046] Public cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0047] Hybrid Cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct entities but are tied together by standardized or proprietary technologies that allow for data and application portability (e.g., cloud bursting for load balancing between clouds).

[0048] A cloud computing environment is a service-oriented approach that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure comprising a network of interconnected nodes. A representative cloud computing node is shown in Figure 2 above. Specifically, a cloud computing node contains computer systems / servers operating in numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, suitable for use with a computer system / server include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems or devices, and the like. A computer system / server may be described in the general context of computer system-executable instructions, such as program modules, being executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer systems / servers may also be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0049] In a typical cloud computing environment, and as shown in Figure 3, a set of function abstraction layers is provided, which include a hardware layer, a software layer, a virtualization layer, a management layer, and a workload layer.

[0050] The hardware and software layer 300 includes hardware and software components. Examples of hardware components include mainframes such as IBM® zSeries® systems, servers based on RISC (Reduced Instruction Set Computer) architecture such as IBM pSeries® systems, IBM xSeries® systems, IBM BladeCenter® systems, storage devices, and networks and networking components. Examples of software components include network application server software, such as IBM WebSphere® application server software, and database software, such as IBM DB2® database software. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide.) The virtualization layer 302 provides an abstraction layer from which the following example virtual entities can be provided: virtual servers, virtual storage, virtual networks including virtual private networks, virtual applications and operating systems, and virtual clients.

[0051] The management layer 304 provides various management functions. For example, resource provisioning provides dynamic procurement of computing resources and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing provides cost management for each resource utilized within the cloud computing environment and billing consumers or organizations for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. A user portal provides access to the cloud computing environment for consumers and system administrators. Service level management provides cloud computing resource allocation and management to meet required service levels. Service level agreement (SLA) planning and fulfillment provides pre-provisioning and procurement of cloud computing resources that forecast future requirements according to SLAs.

[0052] The workload tier 306 provides functionality for which a cloud computing environment may be utilized. Examples of workloads and functionality that may be provided from this tier include mapping and navigation, software development and lifecycle management, virtual classroom education delivery, data analytics processing, transaction processing, private cloud implementation of enterprise-specific functionality, and distributed machine learning 308 according to the present disclosure.

[0053] Thus, a typical cloud computing environment has a set of high-level functional components, including a front-end identity manager, business support services (BSS) functional components, operational support services (OSS) functional components, and compute cloud components. The identity manager is responsible for interfacing with requesting clients to provide identity management, and this component may be implemented with one or more known systems, such as Tivoli Federated Identity Manager (TFIM), available from IBM Corporation, Armonk, New York. In appropriate circumstances, TFIM may be used to provide federated single sign-on (F-SSO) to other cloud components. The business support services department provides specific administrative functions, such as billing support. The operational support services component is used to provide provisioning and management of other cloud components, such as virtual machine (VM) instances. A virtual machine is an operating system or application environment that mimics a hardware machine but is installed in software. Cloud components represent the primary computing resources and are typically multiple virtual machine instances used to run target applications accessible via the cloud. One or more databases are used to store directories, logs, and other operational data. All of these components (including the front-end identity manager) are located "in" the cloud, although this is not a requirement. In alternative embodiments, the identity manager may be operated outside the cloud. The service provider may also be operated outside the cloud.

[0054] Some clouds are based on non-traditional IP networks. Thus, for example, a cloud may be based on a two-tier CLOS-based network with special single-tier IP routing that uses MAC address hashing. The techniques described herein may be used in such non-traditional clouds.

[0055] Generally speaking, cloud computing infrastructure provides a virtual machine hosting environment consisting of host machines (e.g., servers or similar physical computing devices) and one or more management servers connected via a network. Typically, each physical server is adapted to dynamically host one or more virtual machines using virtualization technologies such as VMware ESX / ESXi. Multiple VMs can be placed on a single host machine and share the host machine's CPU, memory, and other resources, thereby increasing utilization of an organization's data center. Among other tasks, the management server monitors the infrastructure and automatically manipulates VM placement, including moving virtual machines between hosts as needed.

[0056] In a non-limiting implementation example, representative platform technologies include, but are not limited to, IBM System x® servers with VMware vSphere 4.1 Update 1 and 5.0.

[0057] [Federated Learning and Threat Modeling] A known approach to distributed machine learning is illustrated in Figure 4. The system comprises a fusion (aggregation) server 400 and a number N of data owners or agents 402, sometimes referred to herein as learning agents. In this embodiment, each learning agent has access to a local dataset d, typically consisting of labeled samples. _i, and wish to train the same machine learning or neural network model. Each agent has its own dataset that it wants to protect and not share with other agents or the aggregation server. In typical operation, the distributed learning process might proceed as follows: In step (1), each agent 402 contacts the aggregation server 400 to obtain hyperparameters for training. In machine learning, hyperparameters are parameters whose values ​​are set before the learning process begins, whereas the values ​​of other parameters are derived through training. Each agent 402 trains the same type of neural network. In a typical example, the model associated with an agent then has a parameter vector A=[p1...p k ] is a parameter vector given to agent i. i In step (2), the i-th agent has its local dataset d _i Such training is typically done by taking a mini-batch, a small subset of the overall training data, and in doing so, the i-th agent will have its parameter vector A i In step (3), each agent 402 sends the obtained parameters to the aggregation server 400, which then fuses each parameter of the vector (typically by calculating an average or weighted average). The average may use different priorities (weights m) for different agents, e.g., if agent i has weight m i , the average parameter vector calculated by the aggregation server is given by the following formula:

number

[0058] The process presented above has been shown to produce the same model, at least for additive loss functions, i.e., metric loss functions (e.g., cross-entropy loss, norm-based loss, binary cross-entropy), as if all data were collected in one place and used to train the model. However, this approach presents a challenge in that it exposes the model to an aggregation server. This server is typically a cloud-hosted service and may be untrustworthy if the data owner / agent itself may not be located in the cloud. As described below, the multi-layered security techniques of the present disclosure address this issue.

[0059] Behind this was the need for a key driver for the emergence of federated learning, which addresses the privacy risks and limitations of centralized training, where training data is collected from all parties and pooled in a central server for training. In FL, training data is maintained in a distributed manner on each participant's local device. Participants (or parties) must agree on a model architecture and maintain a local training pipeline. Instead of providing raw training data to a central server, each party trains a local model with its private training data and uploads model updates to the central server, as shown in Figure 4. An aggregator is typically also responsible for managing parties, coordinating training tasks, and merging model updates. Aggregated model updates are dispatched to parties to (typically) synchronize their local models after each training iteration.

[0060] Below we provide some additional background on training DNNs in a federated learning distributed setting. Let θ be the model parameters and L be the loss function. Each party provides its own training data / label pairs (x i ,y i ) for each batch of data. θ L θ (x i ,y i ) to the aggregator, which computes the gradient sum of all parties and locally synchronizes the model parameters of the parties.

number

number

number

[0061] Both the FedSGD and FedAvg algorithms involve only dual sum and average operations. Simply put, if the model is represented as an array, these fusion algorithms perform coordinate fusion across parties. That is, they add or average the parameters at index i in party P1's model M1 with the parameters at the same index i in party P2's model M2. Here, parameters at index i between parties can be fused without knowing the parameters at other indexes. In this way, the entire model update can be split into multiple pieces, deployed to multiple servers, and the same fusion algorithm can be run independently. Furthermore, it is possible to shuffle parameters or gradients before aggregation, as long as all parties use the same reordering. In an FL privacy attack, the completeness of the model update and the data order are crucial to the optimization procedure for reconstructing the training data. A lack of either will lead to a failed reconstruction. As we will explain later, the technique described herein only requires that the parties be able to locally reverse the splitting and reordering; there is no such restriction.

[0062] In general, federated learning can be used in both cross-device and cross-silo scenarios. Cross-device FL training typically uses a large number of mobile or Internet of Things (IoT) devices as clients. Client reliability is extremely low. Devices may join and leave frequently, and they are often battery-powered, resulting in energy constraints. However, cross-silo FL training typically involves a certain number of organizations sharing incentives to collaboratively train models together. Trusted local training facilities can be provided. Therefore, an aggregator can maintain the state of the parties and address them by their unique IDs. Cross-silo training has strict requirements for data confidentiality, placing a greater focus on data privacy. As will be seen below, while the approach presented here addresses the challenges of FL cross-silo training, it can also be applied to the cross-device domain.

[0063] Our threat model assumes an honest-but-curious aggregation server. We assume that all parties involved in the FL training process are benign but do not tend to share training data with each other. Adversaries attempt to inspect model updates uploaded by the parties. Their goal is to reconstruct the training data of the parties participating in the FL training. This threat model is the same as that in FL privacy attacks. We further assume that the parties involved in the FL trust the system-on-chip (SoC) hardware and the EVM that holds the model aggregation workload.

[0064] [System Design] Below, we detail a representative design of the disclosed federated learning framework and explain how this approach effectively mitigates the information leakage vectors of FL privacy attacks. As described above, this framework preferably utilizes a multi-layered security approach, including (1) trusted aggregation, (2) distributed aggregation, and (3) dynamic reordering. The first aggregation technique enables confidentiality and trusted aggregation, preferably via a remotely verifiable encrypted virtual machine (e.g., AMD® SEV EVM) with runtime memory encryption. Secure Encrypted Virtualization (SEV), used in this exemplary embodiment, is a computing technology released by AMD in 2016. It aims to protect security-sensitive workloads in public cloud environments. SEV relies on AMD Secure Memory Encryption (SME) to enable runtime memory encryption. Combined with the AMD Virtualization (AMD-V) architecture, SEV can enforce cryptographic isolation between guest VMs and the hypervisor. Therefore, SEV can prevent higher-privileged system administrators, such as hypervisor-level administrators, from accessing data within the encrypted virtual machine domain. When SEV is enabled, the SEV hardware attaches an Address Space Identifier (ASID) to all code and data in the VM, and this ASID is associated with a separate ephemeral Advanced Encryption Standard (AES) key called the VM Encryption Key (VEK). Key management is performed by the AMD SP, a 32-bit ARM Cortex-A5 microcontroller integrated into the AMD SoC. Runtime memory encryption is performed via the on-die memory controllers. Each memory controller contains an AES engine that encrypts and decrypts data as it is written to main memory or loaded into the SoC. Memory page encryption is controlled via page tables. Physical address bit 47, commonly known as the C bit, is used to mark whether a memory page is encrypted.Like other TEEs, SEV also provides a remote attestation mechanism for attesting the hardware platform and authenticating guest VMs launched by Tobii. Platform authenticity is attested by identity keys signed by AMD and the platform owner. Guest VM owners verify both the authenticity of SEV-enabled hardware and UEFI firmware measurements before provisioning any secrets to assist in booting encrypted virtual machines.

[0065] FIG. 5 illustrates this general confidential aggregation concept. Assume there exists an isolated, independent, trusted execution environment in the cloud, an example of which is shown at 500, such as the cloud execution environment described above with respect to FIG. 3. A TEE, such as TEE 500, allows users to trust a CPU package and outsource computation to a third-party cloud server. Representative TEE technologies include, for example, Intel® Software Guard Extensions (SGX) / Trust Domain Extensions (TDX), AMD® SEV (discussed above), IBM® Protected Execution Facility (PEF), and ARM TrustZone. TEEs are particularly attractive for collaborative ML computations, which may involve large amounts of privacy-sensitive training data, multiple distribution parties, and where stricter data protection is regulated. Here, and as described, TEE 500 serves as a trusted intermediary to isolate the aggregator execution entity from other such execution entities that facilitate federated learning.

[0066] As shown in FIG. 5, the TEE 500 runs an isolated virtual machine associated with an operating system-based container mechanism (e.g., an open-source container such as Kata Container) for packing and deployment. Specifically, the TEE 500 runs the aggregator 502 within an encrypted virtual machine (EVM) 504 supported by runtime memory encryption 506 (e.g., SEV). Running the aggregator within the TEE mitigates memory corruption attacks. The aggregator 502 is one of a set of distributed aggregators that together constitute the single aggregator shown in FIG. 4. Each aggregator, such as the aggregator 502, runs within the EVM 504, and the EVM memory is protected by a separate ephemeral virtual machine encryption key (VEK). In this way, the confidentiality of model aggregation computations is also protected from unauthorized users, such as system administrators, and privileged software, such as the hypervisor or OS, running on the host server. As described below, each party 508 in the federated learning remotely authenticates genuine SEV hardware / firmware before participating in training and establishes an end-to-end secure path for exchanging model updates. Specifically, remote attestation facilitated by the attestation server 505 is used to provide hardware authentication and load-time integrity checking of the aggregator. Also, as shown, each party 508 has data and its own machine learning (ML) infrastructure 510 and typically collaborates by exchanging attributes (e.g., model gradients).

[0067] With this as background, a representative deployment example of the disclosed federated learning framework is now described in more detail.

[0068] Figure 6 shows a typical deployment example for a federated learning framework. In this example, there are four parties 600 (party numbers 1 to 4) participating in federated learning, and the aggregation mechanism is distributed across three aggregator execution entities 602 (aggregator numbers 1 to 3). Each aggregator execution entity 602 runs within a TEE 604, resulting in three TEEs (TEE numbers 1 to 3). Similar to traditional federated learning, in our approach, each party 600 must register with the aggregator 602 to participate in training. Before registering, each party must verify the TEE platform, for example, through remote attestation. First, one aggregator execution entity notifies all parties to initiate the training process. During the training phase, the aggregator performs multiple training iterations with all parties. In each training iteration, each party first synchronizes its local model by downloading the latest model update from the aggregator, then generates a new model update using the local training data and uploads it to the aggregator. The aggregator merges the model updates from all parties and dispatches the aggregated version back to all parties. Global training terminates when a predetermined training criterion is met, e.g., when FL training reaches a specified number of training iterations, or a party can decide to terminate FL training when model accuracy requirements are met locally. Unlike traditional FL, this deployment involves multiple aggregators 602 running within a TEE 604, rather than a single central aggregator as shown in FIG. 4. In this system, aggregators 602 must communicate with each other for training synchronization. Additionally, an attestation server 606 is also deployed, responsible for attesting the workload and provisioning secrets of the aggregators, as will be described below.

[0069] [Trusted Aggregation] As previously mentioned, model updates exchanged between parties and aggregators may contain essential information for reverse engineering of private training data. The following techniques are used to eliminate avenues for adversaries to intercept and inspect model updates in transit and in use. In this design, cryptographic isolation for FL aggregation is enforced, preferably via a mechanism such as (but not limited to) SEV. As noted in FIG. 5, aggregators run within EVMs, and each EVM's memory is protected with a separate ephemeral VEK. In the embodiment shown in FIG. 6, establishing trust between aggregators 602 and parties 600 is divided into two stages:

[0070] Phase I: Startup of the Trusted Aggregator First, the SEV EVM boots securely with the aggregator running inside. To establish trust in the EVM, attestations are provided that prove (1) the platform is an authenticated security (e.g., AMD SEV-enabled) hardware that provides the required security characteristics, and (2) the Unified Extensible Firmware Image (UEFI) used to boot the EVM has not been tampered with. Once remote attestation is complete, a secret is provided to the EVM, preferably as a unique identifier for the trusted aggregator. The secret is installed in the EVM's encrypted physical memory and is used for aggregator authentication in Phase II, described below. In Figure 6, step (1) shows the attestation server 606 facilitating remote attestation. For this purpose, the EVM owner instructs the service provider (e.g., AMD® SP) to transfer a certificate chain, for example, from the platform Diffie-Hellman public key (PDH) to a root (e.g., AMD root key (ARK)). This certificate chain can be verified by the root certificate. Additionally, along with the certificate chain, the UEFI image digest, SEV API version, and VM deployment policy are preferably included in the attestation report.

[0071] The attestation report is sent to the attestation server 606, which includes the root certificate, to verify the certificate chain to authenticate the hardware platform. The attestation server 606 then generates a boot blob and a Guest Owner Diffie-Hellman Public Key (GODH) certificate. These are sent back to the aggregation server 606 to negotiate a Transport Encryption Key (TEK) and a Transport Integrity Key (TIK) via Diffie-Hellman Key Exchange (DHKE) and start the EVM. UEFI measurements can be obtained from the SP by suspending the EVM during boot. These measurements are sent to the attestation server 606 to attest to the integrity of the UEFI boot process. Only then does the attestation server 606 generate a packaged secret, preferably including an ECDSA prime251v1 key. The hypervisor (not shown) then introduces this secret into the EVM's physical memory space as a unique identifier for the trusted aggregator and continues the boot process. This secret installation procedure preferably follows a remote attestation protocol, such as the first generation SEV remote attestation protocol. Other remote attestation protocols, such as the upcoming SEV-SNP, may be implemented to further increase the integrity of the activation process.

[0072] Phase II: Aggregator Certification Parties participating in an FL must ensure that they are interacting with a trusted aggregator with runtime memory encryption protection. To enable aggregator authentication, and as described above, in Phase I, the attestation server 606 provides an ECDSA private key as a secret during EVM deployment. This key is used to sign challenge requests and thus serves to identify legitimate aggregators. In step (2) of FIG. 6, before participating in an FL, a party first attests to its aggregator by participating in a challenge request protocol. To this end, the party 600 sends a randomly generated nonce to the aggregator 602. The aggregator 602 digitally signs the nonce with its corresponding ECDSA private key and then returns the signed nonce to the requesting party. The party verifies that the nonce is signed with the corresponding ECDSA public key. If the verification is successful, the party 600 then proceeds to register with the aggregator 602 to participate in an FL. Furthermore, a secure path is preferably provided to protect communications between aggregators and between the aggregators and the parties for updating model parameters. The secure channel may be implemented using Transport Layer Security (TLS) to support mutual authentication between the parties and the aggregator. In this way, all exchanged model updates are protected both in use and in transit.

[0073] [Distributed aggregation with model division] While enabling trusted aggregation is a significant advantage, it may not be sufficient because there is no guarantee that the TEE is omnipotent and will not reveal security vulnerabilities in the future. Therefore, a second layer of security, distributed aggregation with model partitioning, increases the fault tolerance of the system so that even if the TEE is compromised with a data leak, an adversary will still not be able to reconstruct the training data from the model updates. This aspect of the disclosure will now be detailed again with respect to the exemplary embodiment shown in FIG. 6.

[0074] As previously explained, each aggregator 602 operates within the EVM and is responsible for only a portion of the model updates. In Figure 6, three aggregators are established, and as previously explained, each participating party authenticates and registers with all aggregators. In this example, distributed aggregation is enabled as follows:

[0075] [Training synchronization between aggregators] Communication paths between the aggregators are maintained, for example, for training synchronization in step (3). Any one of the aggregators 602 can start a training iteration and becomes the initiator node by default. All other aggregators become follower nodes and wait for commands from the initiator. In each training iteration, the initiator first starts local training and queries all parties to obtain model updates for fusion. The initiator then notifies all follower nodes to obtain corresponding model updates, aggregate them together, and distribute the aggregated updates back to the parties.

[0076] Distributed aggregation increases the cost of fraudulently obtaining model information at the aggregation point. Aggregators no longer maintain architectural information about the model; they only see vectors of numbers. Furthermore, missing even a very small portion of model updates can render data reconstruction attacks completely ineffective. Therefore, this protection scheme requires the compromise of all TEE-protected aggregators to obtain the complete set of model updates.

[0077] While compromising any TEE-protected aggregator is extremely difficult, below we describe a third layer of safety, dynamic reordering, that can be implemented to further protect federated learning from information leakage or other compromises.

[0078] Dynamic Sorting To this end, and further to obfuscate the information transferred from the parties to the aggregator, a dynamic reordering scheme is preferably deployed that shuffles the partitioned model updates every training iteration (or other defined period). As mentioned above, the dynamic reordering scheme is based on the observation that the order of parameters in the model updates is irrelevant to the fusion algorithm, but is crucial to the data reconstruction algorithm used in FL privacy attacks. Because this data order is obfuscated, it is infeasible for an adversary to generate reconstructed training data, even if they obtain all of the model updates.

[0079] Randomized Model Split

[0080] Model splitting and dynamic reordering are illustrated in Figure 7, with three aggregators shown there. In particular, aggregator 702 (aggregators 1-3) corresponds to aggregator 602 in Figure 6. Before training begins, an aggregator mapper 710 (a data structure) is randomly generated for each DNN model to be trained. The proportion of model parameters for each aggregator is selected by the parties, but can also be set to a default. The local parties must also agree on mapper 710, which is then shared by all parties participating in the FL training. In Figure 7, the first party has trained local model 712. As shown in Figure 7, and using mapper 710, the k parameters of local model 712 are mapped to three aggregators, i.e., aggregators {1-3}, as shown. The shading and cross-hatching represent the aggregator attributes for each parameter in the model. Also shown, model updates are decomposed and reassigned to different aggregators (step (4) in Figure 6), generating a shuffled split. The shuffled splits are then uploaded to their respective aggregators, where merging occurs to produce an aggregated split. After the parties receive the aggregated model updates from the different aggregators, they unshuffle the aggregated model updates into the correct order. The same mapper 710 is then queried again to merge the model updates back into their original positions in the local model (step (5) in Figure 6). While Figure 7 shows only one local model (the trained, merged one), each party has its own configuration of such a local model.

[0081] Preferably, this dynamic reordering scheme shuffles the split model updates for each training iteration. Each reordering is seeded with a secret agreed upon among all parties (e.g., disseminated via a trusted intermediary) and a dynamically generated training iteration ID. Thus, the reordering changes for each training iteration but is preferably the same for all parties. In other words, the approach herein preferably uses deterministic reordering to dynamically shuffle model updates for each training iteration. An aggregator (i.e., an aggregator execution entity) merges the model updates, and the parties are responsible for restoring the order of the aggregated model updates. This approach mitigates data leakage attacks by dynamically shuffling the order of uploaded model parameters. Thus, this approach has advantages, such as more efficient privacy protection for local training data in federated learning.

[0082] Thus, according to this aspect, the parties preferably split the entire model update they generate into multiple parts (splits) and deploy the splits to multiple servers (aggregation execution entities) where the same fusion algorithm runs independently. Furthermore, parameters or gradients (more generally, elements) may be shuffled (i.e., permuted) locally before aggregation, as long as all parties perform the same permutation. Parties are only required to be able to reverse the splitting and permutation locally. Local model update splitting and permutation can be performed periodically, for example, at each training iteration or at some other defined periodicity; alternatively, update splitting and permutation occur asynchronously.

[0083] The partitioning of the entire local model into partitions and the reordering of one or more elements within each partition occurs for each modeling iteration. All partitions either apply the same partitioning or none at all. Furthermore, partitioning or partitioning strategies, or a combination thereof, can be applied to a centralized aggregator (when the number of aggregating entities is equal to one). In such a case, there is no partitioning, but there is weight reordering in the model update.

[0084] In general, the dynamic reordering scheme described above facilitates aggregate obfuscation in federated learning. A party in federated learning (more generally, a first system among multiple systems involved in federated learning) determines that it needs to send an update vector (more generally, an update) for fusion. An obfuscation algorithm is then applied to obfuscate the update vector, generating an obfuscated update vector. A secret shared by each party can be used for this purpose, and each party in the federated learning locally applies the obfuscation algorithm using the same shared secret. As illustrated by example in FIG. 7, a preferred obfuscation algorithm swaps the order of elements in the update vector to generate an obfuscated update vector. As used herein, the concept of swapping the order of elements is synonymous with shuffling or reordering. The obfuscated update vector is then sent to each aggregator execution entity (if multiple such execution entities are used to create the machine learning model). This update can be performed with a different order of elements for each training iteration. While shuffling the order of update vector elements is a preferred technique for obfuscation, other obfuscation algorithms may also be used.

[0085] The techniques described herein offer significant advantages. As those skilled in the art will appreciate, this approach secures federated learning aggregation from reverse engineering attacks while maintaining low overhead and supporting many different deep learning models and frameworks. Furthermore, the techniques described herein provide multiple, structured, and randomized model partitioning mechanisms for decomposing exchanged model parameters. In this way, even if some aggregators are compromised, adversaries can still be prevented from reconstructing training data information. Furthermore, the techniques described herein allow learning participants to authenticate trusted hardware platforms and attest to the workloads being subjected to federated learning, thus further ensuring that sensitive data is not exposed and transmitted without end-to-end cryptographic protection. The techniques described herein do not impact final model accuracy and convergence rates compared to traditional FL training. At the same time, this approach significantly reduces information exposure between non-essential parties and aggregators, which is crucial for implementing FL privacy attacks.

[0086] As described above, the approach herein takes advantage of the unique computational properties of federated learning fusion algorithms and provides architecture and protocol extensions to mitigate potential information leakage paths. The described federated learning system preferably utilizes a three-tiered security strategy: confidential and reliable aggregation, distributed model partitioning, and dynamic reordering of model updates. A federated learning system that implements these security strategies is immune to training data reconfiguration attacks.

[0087] Furthermore, while it is desirable to use all three techniques together, this is not required. Therefore, a federated learning framework implementing the techniques of the present disclosure can benefit from one or more of the following techniques and strategies. The first strategy is to leverage classified computation techniques to enable reliable, remotely provably model aggregation. The second strategy is to distribute a single aggregator across multiple independently executing entities, each preferably having only a fragmented view of the model updates and unaware of the model architecture. The third strategy is to provide support for randomized and dynamic reordering of the split model updates in each training iteration, making data reconstruction algorithms infeasible. By implementing all three layers of security strategies, as described above, the system defeats state-of-the-art federated learning privacy attacks and exhibits low performance overhead in real-world deployments.

[0088] There are further advantages. First, the distributed learning approach does not require the generation of auxiliary inputs, and training participants share only a portion of the obfuscated model parameters during the federated learning process. The trusted execution environment protects the confidentiality of model updates during transmission and aggregation. Furthermore, our approach prevents malicious or compromised aggregators from reconstructing the training data of federated participants. This approach prevents both (i) honest-but-curious aggregators and (ii) malicious or compromised aggregators from reconstructing private training data from model updates. Another advantage is that by using the same approach for different FL tasks, we can achieve training performance comparable to the baseline.

[0089] The above techniques may be implemented using any machine learning algorithm or calculation that can be distributed in the manner described.

[0090] This subject matter may be implemented in whole or in part as a service. Generalizing, the trusted distributed aggregation for federated learning functionality may be provided as a standalone feature or may leverage functionality from other ML-based products and services. For example, the safety techniques herein may leverage known offerings and solutions, such as IBM's Framework for Federated Learning (FFL), to support the described trusted aggregation, distributed multi-aggregation with model partitioning, and permutations of dynamic model updates. Preferably, the aggregator application is containerized to facilitate its deployment, although this is not a requirement. Kata Containers may be utilized to deploy aggregator containers within lightweight VMs. Thus, as previously described, each aggregator container preferably runs in a SEV-protected EVM (or equivalent). To provide the TEE safety functionality, and in this exemplary but non-limiting embodiment, an AMD EPYC 7642 (Rome) microprocessor running firmware SEV APIs is used.

[0091] All or part of the functionality described above may be implemented in a stand-alone approach, e.g., as software-based functionality executed by a hardware processor, or may be available as a managed service (including as a web service via a SOAP / XML interface). Details of specific hardware and software implementations described herein are for illustrative purposes only and do not limit the scope of the described subject matter.

[0092] More generally, each computing device within the context of the disclosed subject matter is a data processing system (as shown in FIG. 2) consisting of hardware and software, and these entities communicate with each other over a network, such as the Internet, an intranet, an extranet, a private network, or any other communications medium or link. Applications on the data processing systems provide native support for Web and other known services and protocols, including, but not limited to, support for HTTP, FTP, SMTP, SOAP, XML, WSDL, UDDI, WSFL, etc. Information about SOAP, WSDL, UDDI, and WSFL is available from the World Wide Web Consortium (W3C), which develops and maintains these standards, and detailed information about HTTP, FTP, SMTP, and XML is available from the Internet Engineering Task Force (IETF). Familiarity with these known standards and protocols is assumed.

[0093] The schemes described herein may be implemented in or in conjunction with a variety of server-side architectures, including simple n-tier architectures, secure systems, web portals, federated systems, etc. Also, as noted above, the techniques herein may be practiced in loosely coupled server (including "cloud" based) environments such as those described in connection with FIG.

[0094] More generally, the subject matter described herein may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both hardware and software elements. In a preferred embodiment, the functionality is implemented in software, including but not limited to firmware, resident software, microcode, etc. Furthermore, as discussed above, the identity context-based access control functionality may take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For purposes of the description, a computer-usable or computer-readable medium may be any apparatus that contains or can store a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus, device). Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disk, and optical disk. Current examples of optical disks include Compact Disk-Read Only Memory (CD-ROM), Compact Disk-Read / Write (CD-R / W), DVD, etc. A computer-readable medium is tangible.

[0095] The computer program product may be a product having program instructions (or program code) for implementing one or more of the described functions. The instructions or code may be stored on a computer-readable storage medium within the data processing system after being downloaded over a network from a remote data processing system. Alternatively, the instructions or code may be stored on a computer-readable storage medium within a server data processing system and adapted to be downloaded over a network to the remote data processing system for use on the computer-readable storage medium within the remote system.

[0096] In a representative embodiment, the fusion server and each agent are implemented in software executed by a special purpose computer, preferably one or more processors. The software is maintained in one or more data stores or memories associated with the one or more processors, and the software may be implemented as one or more computer programs. This special purpose hardware and software collectively provide the functionality described above.

[0097] While the above describes a particular order of operations performed by certain embodiments of the present invention, it should be understood that such orders are exemplary, as alternative embodiments may perform operations in a different order, combine certain operations, overlap certain operations, etc. Reference to certain embodiments herein indicates that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments necessarily include the particular feature, structure, or characteristic.

[0098] Finally, although certain components of the system have been described separately, those skilled in the art should understand that some of the functionality may be combined or shared among certain instructions, program sequences, code portions, etc.

[0099] The techniques herein provide improvements to other technologies or technical areas, such as machine learning systems, security incident and event management (SIEM) systems, other security systems, and automation-based cyber security analytics.

[0100] Having explained the subject matter, we argue that:

Claims

1. 1. A method for federated learning with reduced information leakage, in which model updates provided by participating parties are fused, comprising: partitioning the local model according to a mapper to generate a set of partitions, the mapper mapping elements of the local model to a set of independent aggregator execution entities that together constitute an aggregator; applying a permutation operation to one or more elements in each of the partitions to generate a shuffled partition; forwarding the shuffled partition as the model update to the set of aggregator execution entities; recovering the fused local model upon receiving fused model updates from the set of aggregator execution entities; A method comprising:

2. 2. The method of claim 1 , wherein recovering the fused local model comprises adjusting the order of one or more elements in the fused model update to generate deshuffled partitions that correspond to the original positions of the partitions in the local model; and merging the deshuffled partitions according to the mapper.

3. The method of claim 1 or 2, further comprising dynamically repeating the reordering operation.

4. The method of claim 3 , wherein the permutation operation is repeated for each training iteration.

5. The method of claim 1 or 2, wherein the reordering operation is performed based on a secret mutually agreed upon by the participating parties, and the mapper is shared by all of the participating parties.

6. The method of claim 1 or 2, wherein the element is one of a model parameter and a model gradient.

7. The method of claim 1 or 2, wherein applying the reordering operation adjusts only a single model parameter.

8. a hardware processor; a computer memory holding computer program instructions that, when executed by the hardware processor, provide federated learning with reduced information leakage, wherein model updates provided by participating parties are fused, the computer program instructions comprising: partitioning the local model according to a mapper to generate a set of partitions, said mapper mapping elements of said local model to a set of independent aggregator execution entities that together constitute an aggregator; applying a permutation operation to one or more elements in each of said partitions to generate shuffled partitions; forwarding the shuffled partition as the model update to the set of aggregator execution entities; recovering the fused local model upon receiving fused model updates from the set of aggregator execution entities; a computer memory configured to execute An apparatus comprising:

9. 9. The apparatus of claim 8, wherein the computer program instructions configured to recover the fused local model further comprise computer program instructions configured to: adjust an order of one or more elements in the fused model update to generate a deshuffled partition that corresponds to an original position of the partition in the local model; and merge the deshuffled partitions according to the mapper.

10. 10. Apparatus according to claim 8 or 9, wherein the computer program instructions dynamically repeat the reordering operation.

11. The apparatus of claim 10 , wherein the permutation operation is repeated for each training iteration.

12. 10. The apparatus of claim 8 or 9, wherein the reordering operation is performed based on a secret mutually agreed upon by the participating parties, and the mapper is shared by all of the participating parties.

13. The apparatus of claim 8 or 9, wherein the element is one of a model parameter and a model gradient.

14. 10. The apparatus of claim 8 or 9, wherein the computer program instructions configured to apply the reordering operation adjust only a single model parameter.

15. 1. A computer program for use in a data processing system for providing federated learning with reduced information leakage, wherein model updates provided by participating parties are fused, the computer program comprising: partitioning the local model according to a mapper to generate a set of partitions, the mapper mapping elements of the local model to a set of independent aggregator execution entities that together constitute an aggregator; applying a permutation operation to one or more elements in each of said partitions to generate shuffled partitions; forwarding the shuffled partitions as the model updates to the set of aggregator execution entities; recovering the fused local model upon receiving fused model updates from the set of aggregator execution entities; A computer program for executing

16. 16. The computer program product of claim 15, wherein recovering the fused local model further comprises: adjusting an order of one or more elements in the fused model update to generate a deshuffled partition that corresponds to the original position of the partition in the local model; and merging the deshuffled partitions according to the mapper.

17. 17. A computer program product as claimed in claim 15 or 16, further causing the data processing system to perform the step of dynamically repeating the sorting operation.

18. 20. The computer program product of claim 17, further causing the data processing system to perform the step of repeating the sorting operation for each training iteration.

19. 17. A computer program product as claimed in claim 15 or 16, wherein the reordering operation is performed based on a secret mutually agreed upon by the participating parties, and the mapper is shared by all of the participating parties.

20. 17. The computer program of claim 15 or 16, wherein the element is one of a model parameter and a model gradient.

21. 17. A computer program product as claimed in claim 15 or 16, wherein applying the reordering operation comprises adjusting only a single model parameter.

22. A federated learning method that is secure against information leakage, comprising: partitioning the aggregator into a set of independent aggregator-executing entities; at a local computing entity associated with a party, the party being one of a set of parties participating in federated learning, and model updates generated by the participating parties being fused at an aggregator; partitioning the local model according to a mapper to generate a set of partitions, the mapper mapping elements of the local model to a set of independent aggregator execution entities that together constitute an aggregator; applying a permutation operation to one or more elements in each of the partitions to generate a shuffled partition; forwarding the shuffled partitions to the set of aggregator execution entities as model updates associated with the parties; A method comprising:

23. In the local computing entity: From the set of aggregator execution entities, receiving a set of fused model updates; recovering the fused local model; 23. The method of claim 22, further comprising:

24. 24. The method of claim 23, wherein the fused local model is recovered by adjusting the order of one or more elements in the fused model update to generate unshuffled partitions that correspond to the original positions of the partitions in the local model, and merging the unshuffled partitions according to the mapper.

25. 25. The method of any one of claims 22 to 24, further comprising repeating the permutation operation for each training iteration.

Images