SYSTEM AND METHOD FOR IMPLEMENTING A FEDERATED LEARNING ENGINE FOR VERTICAL AND HORIZONTAL AI INTEGRATION - Patent application

Abstract

Disclosed herein is a system and method for implementing a federated learning engine for integrating vertical and horizontal AI. The method can include receiving a global model including multiple layers from a central aggregator communicatively coupled to multiple user environments. The method can include training a minimal model on the global model with data collected in the user environments, uploading at least a portion of the minimal model to the central aggregator, receiving the multiple minimal models, and creating a fusion model based on the multiple minimal models received.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. patent application Ser. No. 17 / 399,911, filed Aug. 11, 2021, and entitled "SYSTEM AND METHOD FOR IMPLEMENTING A FEDERATED LEARNING ENGINE FOR VERTICAL AND HORIZONTAL AI INTEGRATION," which is incorporated herein by reference in its entirety. [Background technology]

[0002] background This application relates to distributed machine learning and federated learning. Summary of the Invention [Means for solving the problem]

[0003] Quick Overview One aspect of the disclosure relates to a method that includes receiving a global model including a plurality of layers from a central aggregator communicatively coupled to a plurality of user environments, training a minimal model on the global model with data collected in the user environments, uploading at least a portion of the minimal model to the central aggregator, receiving the plurality of minimal models, and creating a fusion model based on the plurality of received minimal models.

[0004] In some embodiments, the multiple user environments include multiple tenancies in a cloud computing network. In some embodiments, the global model may be a deep learning model. In some embodiments, the deep learning model may be a Transformer. In some embodiments, the deep learning model may be a Bidirectional Encoding Representation with Transformers (BERT) model.

[0005] In some embodiments, the minimal model can incorporate an output of the global model. In some embodiments, the output of the global model includes layer outputs of at least some of the layers of the global model. In some embodiments, uploading the minimal model to the central aggregator includes removing a classifier head from the minimal model. In some embodiments, uploading the minimal model to the central aggregator includes uploading a binary of the minimal model.

[0006] In some embodiments, a plurality of minimal models are received from a central aggregator. In some embodiments, creating the fusion model based on the received plurality of minimal models includes training the fusion model on the received minimal models. In some embodiments, creating the fusion model based on the received plurality of minimal models includes combining data representing layers and weights from each of the plurality of minimal models.

[0007] In some embodiments, creating a fusion model based on the received plurality of minimal models includes generating a classifier head and applying the classifier head to a layer of the fusion model. In some embodiments, the method includes receiving data for use in generating a machine learning model output, ingesting the data with a global model in a user environment, collecting intermediate outputs of the global model, ingesting the intermediate outputs with the fusion model, and outputting a prediction using the fusion model. In some embodiments, the method includes receiving data for use in generating a machine learning model output, ingesting the data with a global model in a user environment, collecting a first intermediate output of the global model, ingesting the first intermediate output of the global model with at least some of the plurality of minimal models, collecting a second intermediate output of at least some of the plurality of minimal models, ingesting the intermediate outputs with the fusion model, and outputting a prediction using the fusion model.

[0008] One aspect of the disclosure relates to a system including a memory and at least one processor capable of receiving a global model from a central aggregator communicatively coupled to a plurality of user environments, the global model including a plurality of layers, training a minimal model on the global model with data collected in the user environments, uploading at least a portion of the minimal model to the central aggregator, receiving a plurality of minimal models, and creating a fusion model based on the plurality of received minimal models.

[0009] In some embodiments, uploading at least a portion of the minimal models to a central aggregator includes removing a classifier head from the minimal models. In some embodiments, creating a fusion model based on the received plurality of minimal models includes training the fusion model on the received minimal models.

[0010] One aspect of the disclosure relates to a non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, which when executed by the one or more processors cause the one or more processors to receive a global model including a plurality of layers from a central aggregator communicatively coupled to a plurality of user environments, train a minimal model on the global model with data collected in the user environments, upload at least a portion of the minimal model to the central aggregator, receive a plurality of minimal models, and create a fused model based on the plurality of received minimal models.

[0011] In some embodiments, uploading at least a portion of the minimal models to a central aggregator includes removing a classifier head from the minimal models. In some embodiments, creating a fusion model based on the received plurality of minimal models includes training the fusion model on the received minimal models. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of one embodiment of an AI aspect. [Diagram 2] FIG. 1 is a schematic diagram of one embodiment of a federated learning system. [Diagram 3] FIG. 1 is a detailed schematic diagram of one embodiment of a federated learning system. [Figure 4] FIG. 1 is a schematic diagram of an embodiment of a process for generating a fusion model output. [Diagram 5] 1 is a flow chart illustrating an embodiment of a process for generating a fusion model. [Figure 6] 1 is a flow chart illustrating an embodiment of a process performed at a customer environment to generate a fusion model. [Figure 7] 1 is a flow chart illustrating an embodiment of a process for generating an output using a fusion model. [Figure 8] FIG. 1 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 9] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 10] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 11] FIG. 1 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 12] FIG. 1 is a block diagram illustrating an example computer system in accordance with at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of particular embodiments. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments or designs.

[0014] Intelligence Systems 1, a schematic diagram of one embodiment of aspects of an intelligence system 151 is shown. Intelligence system 151 illustrates connections between data sources 150, data processing 152, enterprise IT 158, AI services 160, intelligent apps / insights modules 168, and enterprise solution customers 170 or service customers 172. In some embodiments, aspects of intelligence system 151 can collect data, train models, and provide models and / or output of the models to one or several customers.

[0015] Intelligence system 151 includes data sources 150. The data sources may include data sources used by intelligence system 151. These may include, for example, Internet of Things devices, social media, mobile apps, smart watches, health apps, or other data sources.

[0016] Data collected from data sources 150 may be provided to a data processing module 152, which may include an ingestion submodule 154 that may provide, for example, real-time and / or batch ingestion to a data governance module 156. The governance module 156 may provide data cataloging, data orchestration, and / or data integration.

[0017] The data processing module 152 can be connected to an enterprise IP 158 that can provide a system of record, enterprise risk management, human capital management, customer relationship management, etc. The enterprise IT 158 can be connected to an AI service 160 that can include a service module 162 that includes one or several services. These one or several services each represent an AI function. These can include, for example, computer vision, speech translation, anomaly detection, language services, predictive services, federated AI services, etc.

[0018] Each service in services module 162 may be a customer endpoint that enables customers to use both pre-trained and custom model intelligence provided via AI services 160. Services module 162 may include the services identified above and may include newer services as they are developed and produced.

[0019] The AI ​​services 160 may include an AI and data science platform 164. The AI ​​and data science platform 164 may provide infrastructure and functionality to support the execution of machine learning and data workloads. This may include, for example, support for training models, support for inference, etc. In some embodiments, the platform 164 may include an orchestration layer, which may be a collection of functionality that encapsulates both the ML infrastructure layer and the service layer from the complexities and idiosyncrasies of those layers for ease of consumption and management within the AI ​​service logic. This layer may include, for example, an asset manager, a training manager, a deployment manager, and / or an orchestration manager.

[0020] Network services 166 can provide infrastructure resources that can be used by individual AI services across multiple AI service customers or shared by multiple AI services across multiple AI service customers. If such security and privacy isolation is required by the AI ​​service customer, multiple AI services can share dedicated ML infrastructure resources for a particular AI service customer. The common infrastructure required is compute (CPU / GPU), network, and storage.

[0021] The system 151 further includes an intelligent apps / insights module 168. The apps in this module enable, for example, faster time to business insights, accelerated processes, increased time to market, and improved customer experience. In some embodiments, these can further reduce costs and improve productivity. In some embodiments, this can be accomplished without the use of data scientists or IT.

[0022] The system 151 can interact with customers 170, 172 directly via the AI ​​service 160 or via an intelligent app / insights module 168.

[0023] Federated Learning Engine Many machine learning approaches utilize training data that is centralized. This training data can be centralized, for example, within a data center or within one or a few machines. Although such centralization of data has proven effective, such centralization can have several drawbacks. These drawbacks include issues arising from the collection and / or management of data that contains personal, protected, and / or confidential information. Additionally, aggregating large amounts of data can require large amounts of hardware and the costs associated with such hardware. Finally, issues associated with catastrophic forgetting may prevent machine learning models from improving in performance as the ensemble of training data grows larger.

[0024] This problem becomes especially challenging when using artificial intelligence models for multiple different markets. For example, a single AI model may have difficulty working effectively in both the healthcare and hospitality markets. For such a model to work equally well in both markets, the model cannot include market-specific training, as this would result in catastrophic forgetting of training relevant to the other market. Thus, many such models lack the specialized training required to deliver a high level of performance.

[0025] For example, a horizontal AI model may not be specific to one or a few industry domains. This horizontal AI model may solve a broad range of problems across many different industries, and in some embodiments, may provide a use-case agnostic program. In some embodiments, horizontal AI may be used to prioritize customer leads, predict which hires will be most successful, recommend products, or target advertisements. Vertical AI is specific to an industrial and may be applied to a particular problem in a particular industry, e.g., highly optimized for that industry. In vertical AI applications, models may be trained utilizing industry-specific types of data from the target industry. In some embodiments, inferences from the vertical AI may provide unique improved predictions and / or insights.

[0026] In some embodiments, the combination of horizontal AI and vertical AI disclosed herein can enable the solving of a wide range of problems and, in some embodiments, can help customers select data for building AI verticals. In some embodiments, the present disclosure can facilitate the rapid training, generation, and / or production of AI models using highly heterogeneous data from multiple sources. The embodiments disclosed herein can accelerate rapid prototyping of industry-specific models, for example, with vertical differentiation in a specific industry domain. The embodiments disclosed herein can enable the use of model learning to optimize infrastructure costs for model training and deployment.

[0027] In one particular embodiment disclosed herein, a horizontal AI in the form of an initial global model can be used in combination with a vertical AI in the form of one or several minimal and / or fused models. Through the combination of horizontal and vertical AI disclosed herein, in some embodiments, the benefits of both horizontal and vertical AI can be achieved simultaneously. In some embodiments, the global model can be updated over time based on the minimal model and / or based on the fused model, and thus, over time, the global model can become a vertical AI with these updates. This targeted vertical model can then be stored and utilized to provide vertical-specific AI without further federated learning.

[0028] Further, the present disclosure relates to a system and method for federated learning, in which a global model is trained centrally and provided to different customers, also referred to herein as users. These users can install and utilize this global model, and based on the data they collect, a supplemental machine learning model, also referred to herein as a minimal model, can be trained. This supplemental machine learning model, in some embodiments, can receive as input one or several outputs of the global model, for example, inter-layer outputs of the global model. Based on these and / or other inputs from the global model, the supplemental model can generate an output.

[0029] Such supplemental models can be provided to a central server where these supplemental models can be stored. These supplemental models can be provided to the central server by first removing the classifier heads and then sending the removed supplemental models to the central server. In some embodiments, the supplemental models can be sent to the central server in a binary format, which can identify layers and / or nodes of the supplemental models and weights associated with the layers and / or nodes of the supplemental models.

[0030] These supplemental models can be received from several users provided with the global model, creating a library of supplemental models at the aggregator. A user can request one or several supplemental models and / or provide one or several supplemental models. Some or all of these one or several supplemental models can be used to create a fusion model. In some embodiments, a fusion model can be created from a combination of supplemental models and / or a fusion model can be trained on the supplemental models. In some embodiments, a fusion model can incorporate data from a minimal model, specifically including layers and / or nodes and their associated weights from a minimal model.

[0031] The use of a pre-trained global model in combination with a minimal model allows a user to quickly incorporate changes and / or updates and allows a user to customize a model for a particular application, for example, without the risk of catastrophic forgetting. Additionally, a user can select minimal models to include in the fused model. These minimal models can be selected based on information related to the minimal models, which can include metadata related to the minimal models. In some embodiments, this metadata can indicate the source of the minimal models, the confidence level and / or accuracy of the minimal models, and / or other information related to the minimal models. Based on this information, a user can decide whether to trust a minimal model and / or select one or several minimal models to include in the fused model.

[0032] In operation, when a user wishes to make a prediction using the fusion model, the user may receive and / or select data, such as feature data. This data may be consumed by a global model of the user's environment. Intermediate outputs of the global model (which may include inter-layer outputs) may be collected by the user. These intermediate outputs of the global model may be features for ingestion into a next machine learning model. In some embodiments, this next machine learning model may be a minimal model from which the fusion model is created. In such an embodiment, intermediate outputs of the minimal model (which may include inter-layer outputs) may be collected by the user. These intermediate outputs of the minimal model may be features for ingestion into the fusion model.

[0033] Some or all of the features of the global model and / or the minimal model from which the fusion model is created may be captured by the fusion model. The fusion model, which may include a classifier head, may generate an output that may be a final output. This output may be a prediction, a classification, etc.

[0034] Referring now to FIG. 2, a schematic diagram of one embodiment of a federated learning system 200 is shown. The federated learning system 200 can include an aggregator 202 and one or more customer environments 212. The aggregator 202 and one or more customer environments 212 can each comprise hardware or virtual instances. Thus, in some embodiments, the aggregator 202 and one or more customer environments 212 can each comprise one or more processors, servers, computers, etc., or can comprise virtual computing instances. In some embodiments, the aggregator 202 and one or more customer environments 212 can all comprise virtual instances, such as on a virtual cloud network. In some embodiments, the aggregator and one or more customer environments 212 can run as separate tenancies in the cloud network. In some embodiments, each of the one or more customer environments 212 can access private data of an associated customer, while in some embodiments the aggregator 202 cannot access the private data of this customer.

[0035] The aggregator 202 and each of the customer environments 212, in some embodiments, may be communicatively linked via a communications network 210. The communications network 210 may enable communication and / or data transfer between the aggregator 202 and one or more customer environments 212. In some embodiments, the communications network 210 may include a wired or wireless network, a local area network, a wide area network, a public network, a private network, etc.

[0036] The aggregator 202 can collect and manage information, which may include, for example, information identifying one or several customers, the current state of a process, recording and monitoring information, etc. In some embodiments, the aggregator 202 manages the information, thereby facilitating the collection, averaging, and distribution of minimal models. In some embodiments, the aggregator 202 can further train and / or update the training of the global model.

[0037] The aggregator 202 may include a federated server 204. The federated server 204 may be implemented in hardware or software. The federated server 204 may generate and / or train a global model, provide the global model to one or more customer environments 212, receive minimal models from one or more customer environments 212, manage storage and tracking of minimal models, generate and / or collect metadata associated with the minimal models, and provide one or more minimal models to one or more customer environments 212. The federated server 204 may be implemented in hardware or software. In some embodiments, the federated server 204 may comprise virtual computing instances and / or services provided as part of a virtual cloud network.

[0038] The aggregator may include a global model store 206. The global model store 206 may store the global models. In some embodiments, this may include storing metadata related to the global models. This metadata may include, for example, versions, training data, etc. The global model store 206 may comprise a database and / or memory. This memory may include allocated portions of memory or may include one or several separate memory devices.

[0039] The global model may be a base model. The global model may include, for example, a neural network, a decision tree, a linear classifier / regression, a deep learning model, a deep reinforcement learning model, such as, for example, a deep deterministic policy gradient (DDPG) or a deep Q-network (DQN). In some embodiments, the global model may comprise a transformer, such as, for example, a Bidirectional Encoding Representation with Transformers (BERT) model. The global model may be trained by an aggregator, specifically a federated server. In some embodiments, the global model may be trained for feature extraction. In some embodiments, for example, the global model may be trained for feature extraction from a given text. These features may include, for example, linguistic features and / or contextual features. In some embodiments, the global model may be trained using data accessible by the aggregator 202, including training data in the global model store 206.

[0040] The aggregator may include a minimal model store 208, also referred to herein as a supplemental model store 208 or a local model store 208. The minimal model store 208 may store one or several minimal models. The minimal model store 208 may store binaries of the minimal model, which may represent layers, nodes, and / or weights of the minimal model. The minimal model store 208, in some embodiments, may store metadata related to the minimal model. This metadata may include, for example, an identification of a source of the minimal model, information for establishing and / or assessing the reliability of the plausibility model, information related to the training of the plausibility model, etc. In some embodiments, the information related to the training of the plausibility model may identify the task for which the plausibility model was trained and / or the particular vertical for which the plausibility model was trained. The minimal model store 208 may comprise a database and / or a memory. The memory may include an allocated portion of memory or may include one or several separate memory devices.

[0041] Each of the customer environments 212 can include a fusion server 214. The fusion server 214 can receive a global model from the aggregator 202, can store the global model, can generate a minimal model on top of the global model, can send the global model to the aggregator 202 for storage in the minimal model store 208, can receive one or more minimal models and / or updates from the aggregator 202, can train and / or build a fusion model based on these one or more minimal models and / or updates received from the aggregator 202, and can generate one or more outputs and / or predictions using the minimal model and / or global model. The fusion server 214 can be implemented in hardware or software. In some embodiments, the fusion server 214 can comprise a virtual computing instance and / or service provided as part of a virtual cloud network.

[0042] The customer environment 212 may include a data store 216. The data store 216 may store customer information for customers associated with the customer environment. In some embodiments, this customer information may be private and may be stored such that the customer information is not accessible by the aggregator 202 and / or other customer environments 212. The customer information in the data store 216 may be used for training the minimal model and for training the fusion model. The data store 216 may comprise a database and / or memory. The memory may include an allocated portion of memory or may include one or several separate memory devices.

[0043] The customer environment 212 can include a model store 218. The model store 218 can, in some embodiments, store a minimal model trained by the customer environment 212 on top of a global model, and in some embodiments, store a fusion model generated and / or trained by the customer environment 212 on top of a received minimal model. The model store 218 can, in some embodiments, further store a global model received by the customer environment 212 from the aggregator 202. The model store 218 can comprise a database and / or memory. The memory can include allocated portions of memory or can include one or several separate memory devices.

[0044] 3, a detailed schematic diagram of one embodiment of a system 200 is shown. In some embodiments, the system 200 can provide a framework for federated learning. The system includes an aggregator 202 and one customer environment 212. As shown in FIG. 2, the system 200 can include multiple additional customer environments 212, although only a single customer environment 212 is shown for purposes of providing more detail about the components and / or modules of each of the aggregator 202 and customer environment 212.

[0045] The aggregator 202 may include a fusion manager 302, a protocol manager 304, and a FL connection manager 306. These modules may be modules within the federation server 204 and / or modules accessible and / or controllable by the federation server 204. The fusion manager 302 may be coupled to a model store, specifically, a global model store 206 and a local model store 208.

[0046] As seen in Figure 3, the local model store 208 may include multiple minimal models 308, also referred to herein as supplemental models 308 or local models 308. These may include a first minimal model 308-A received from a first customer environment 212, such as, for example, customer environment A 212-A shown in Figure 2, and a second minimal model 308-B received from a second customer environment 212, such as, for example, customer environment B 212-B shown in Figures 2 and 3. The local model store 208 may include a binary for each of the minimal models 308 included in the local model store 208, which may identify the layer, node, and / or weighting value of the model from which the minimal model was created.

[0047] 3, the global model store 206 can include a global model 310. The global model 310 can be trained and / or generated by the aggregator 202, and specifically, in some embodiments, by the fusion manager 302.

[0048] The fusion manager 302 can select a particular model from one of the model stores 206, 208. In some embodiments, the fusion manager 302 can provide information to and / or retrieve information from the model stores 206, 208. This can include providing the global model 310 to the global model store 206 and / or providing one or some of the minimal models 308 to the minimal model store 208. In some embodiments, the fusion manager 302 can retrieve models from the model stores 206, 208. This can include retrieving the global model 310 from the global model store 206 so that the global model can be provided to one or more of the customer environments 212. In some embodiments, this can include retrieving one or some of the minimal models 308 from the minimal model store 208. These minimal models 308 can then be provided to one or more customer environments 212. In some embodiments, the fusion manager 302 can retrieve and / or provide information regarding model performance. Thus, in some embodiments, the fusion manager 302 can access metadata associated with models stored in one of the model stores 206, 208. The fusion manager can be implemented in hardware or software, and in some embodiments can be part of the federation server 204.

[0049] The Protocol Manager 304 facilitates communication between the aggregator 202 and / or federated server 204 and one or more of the customer environments 212. The Protocol Manager 304 may further provide governance over message exchanges, e.g., learning protocols, between the customer environments 212 and the aggregator 202 and / or federated learning server. These messages may include, for example, queries, model updates, establishing FL configurations, registering new customers, etc.

[0050] The FL connection manager 306 can act as a connection interface between the aggregator 202 and the customer environments 212. In some embodiments, the FL connection manager 306 can establish and manage the networking between the various components necessary for the operation of the system 200. In particular, the FL connection manager can establish and manage the communications and / or network between the aggregator 202 and one or several customer environments 212. In some embodiments, the FL connection manager can support low-level APIs from the OSI / TCP-IP stack.

[0051] The customer environment 212 can include a local training manager 312, a protocol manager 314, and a FL connection manager 316. These modules may be modules within the federated server 204 and / or modules accessible and / or controllable by the federated server 204.

[0052] The local training manager 312 can train machine learning models in the customer environment 212. This may include training a minimal model on a global model and / or training and / or building a fusion model. The local training manager 312 can be communicatively coupled to the local data manager 318 and the model store 218. The local data manager 318 can be communicatively coupled to the data store 216. The local data manager 318 can access and provide access to the data in the data store 216. This data in the data store 216 of the customer environment 212 can, in some embodiments, be isolated to the particular customer environment 212 in which it is contained.

[0053] By accessing the data in the data store 216, the local training manager 318 can train, generate, and / or build machine learning models. In particular, the local training manager 318 can train, generate, and / or build minimal and / or fused models. These models can be stored in the model store 218.

[0054] Similar to the aggregator 202, the customer environment 212 can include a protocol manager 314 and a FL connection manager 316. The protocol manager 314 facilitates communication between the customer environment 212 and the aggregator 202. In some embodiments, the protocol manager 314 can further facilitate communication with other customer environments 212. The protocol manager 314 can further provide governance over message exchanges between the customer environments 212 and / or between the customer environments and the aggregator 202 and / or the federated learning server 204. These messages can include, for example, queries, model updates, establishing FL configurations, registering new customers, etc.

[0055] The FL connection manager 316 can serve as a connectivity interface between a customer environment 212 and an aggregator 202, and / or between a set of customer environments 212. In some embodiments, the FL connection manager 316 can establish and manage networking between various components necessary for the operation of the system 200. In particular, the FL connection manager can establish and manage communications and / or networks between a customer environment 212 and an aggregator 202, and / or between a set of customer environments 212. In some embodiments, the FL connection manager can support low level APIs from an OSI / TCP-IP stack.

[0056] Referring now to FIG. 4, a schematic diagram of one embodiment of a process 400 performed by a customer environment in generating a prediction is shown. As seen in FIG. 4, in some embodiments, a prediction may be made utilizing multiple models. These models may be stored in the customer environment 212. User input 401 is fed into the models. In some embodiments, this may include feeding the user input 401 into a global model 402. The global model 402 may comprise multiple layers 404-A through 404-N. In some embodiments, the global model 402 may operate as a feature extractor, where the global model 402 may provide multiple outputs 406. These outputs are referred to herein as intermediate outputs, as they are used as features by subsequent models.

[0057] In some embodiments, the global model 402 may provide outputs after classification and / or processing by some or all of the layers. Thus, for example, the global model 402 may provide a first output after processing and / or classification of layer 404-A, a second output after processing and / or classification of layer 404-B, a third output after processing and / or classification of layer 404-C, etc., up to the Nth output after processing and / or classification of layer 404-N.

[0058] These intermediate outputs 406 can be consumed by a subsequent model, such as a minimal model or a fused model. In some embodiments, the minimal model and the fused model are configured to consume the output of the global model, which may include the layer outputs of at least some of the layers of the global model. In some embodiments, these intermediate outputs 406 can be consumed by one or several minimal models 408 stored in the model store 218 of the customer environment 212, and in some embodiments, these intermediate outputs 406 can be consumed by the fused model 414.

[0059] 4, the intermediate outputs from the global model 402 can be consumed by multiple minimal models 408 in the model store 218 of the customer environment 212. Each of these minimal models 408 can act as a feature extractor and, in some embodiments, can provide multiple outputs. These outputs are referred to herein as intermediate outputs because they are used as features by subsequent models.

[0060] In some embodiments, the minimal model 408 may provide outputs 412 after classification and / or processing through some or all of the layers. Specifically, in some embodiments, outputs 412 may be provided after each of some or all of the layers is completed. Thus, in some embodiments, the minimal model may provide up to as many outputs 412 as there are layers in the minimal model.

[0061] These outputs may be received and fed into a fusion model 414, which may comprise multiple layers 416 and a classifier head 418. The fusion model 414, and specifically the classifier head, may provide a final output in the form of a desired task output 420.

[0062] 5, a flow chart is shown illustrating one embodiment of a process 500 for generating a fusion model. Process 500 may be performed by all or a portion of federated learning system 200. Process 500 begins at block 502, where aggregator 202, and specifically federated server 204, receives and / or obtains global training data. In some embodiments, this may include receiving and / or obtaining the global training data from global model store 206.

[0063] At block 504, a global model is trained. In some embodiments, the global model may be trained by the aggregator 202, specifically by the federation server 204 and / or the fusion manager 302. The global model may be trained using global training data. After the global model is trained, the global model may be stored in the global model store 206.

[0064] At block 506, the global model is provided to the customer environments 212. The global model may be provided to the customer environments by the aggregator 202 via the federation server 204, specifically via the FL connection manager 306 and the protocol manager 304. In some embodiments, the aggregator 202 may send the global model to some or all of the customer environments 212 to which the aggregator is connected. The global model is received by the customer environment 212, which stores the global model in its model store 218.

[0065] At block 508, local data is collected at each of the customer environments 212. The local data collected by a customer environment 212 may be stored in a data store 216 for that customer environment 212. The local data may be stored in the data store 216 by a local data manager 318 of the customer environment 212.

[0066] At block 510, some or all of the customer environments 212 that received the global model may train a minimal model. In some embodiments, this may include training a minimal model on the global model.

[0067] The minimal model trained by a customer environment 212 may be trained by a local training manager 312 of that customer environment 212. The minimal model of a customer environment 212 may be trained using local data of that customer environment 212. This local data may be obtained from the data store 216 of that customer environment 212 by a local data manager 318 of that customer environment 212. Execution of the steps of block 510 by multiple customer environments 212 may result in the creation of multiple minimal models, each of which may be trained using local data of the customer environment 212 on which the minimal model is trained. The minimal models generated by a customer environment 212 may be stored in the model store 218 of that customer environment 212.

[0068] At block 512, the customer environment removes the head layer from the trained minimal model to create a pruned minimal model, which may include the customer environment 212 deleting the classifier head of the minimal model of the customer environment 212. In some embodiments, the pruned minimal model may include a binary of the minimal model, which may identify, for example, layers and / or nodes of the supplemental model and weights associated with the layers and / or nodes of the supplemental model.

[0069] After the head layer of the minimal model is removed, the minimal model to be removed may be uploaded by the customer environment to the aggregator 202, i.e., the binary of the minimal model may be uploaded. This may include the protocol manager 314 and / or the FL connection manager 316 of the customer environment 212 communicating the minimal model to be removed to the connection manager 306 and / or the protocol manager 304 of the aggregator 202. The aggregator 202 may receive the minimal model to be removed from the customer environment 212 and may store the minimal model to be removed in the global model store 206.

[0070] At block 514, the received minimal models to be pruned are evaluated by the aggregator 202. The aggregator 202 may generate minimal model scores and may associate the minimal models with their minimal model stores. These minimal model scores may characterize one or more attributes of the minimal models, the training of the minimal models, the customer environments 212 that trained the minimal models, etc. In some embodiments, these minimal model scores may reflect, for example, the reliability of the associated minimal models. In some embodiments, the minimal model scores of the minimal models may be generated based on metadata associated with the minimal models.

[0071] At block 516, the customer environment selects, fetches, requests, and / or receives one or more minimal models to be pruned from the aggregator. In some embodiments, the customer may request information characterizing one or more minimal models to be pruned via the customer environment. This information may include, for example, a score of the minimal model, metadata associated with the minimal model, etc. In some embodiments, the customer may select and / or request one or more minimal models via the customer environment. In some embodiments, these one or more minimal models may be selected based on this information characterizing one or more minimal models to be pruned. For example, the customer may select and / or request one or more minimal models if the requested one or more minimal models have information indicative of one or more desired attributes, such as, for example, sufficiently high confidence, training of the minimal model for one or more desired tasks, etc. In some embodiments, the customer may select the minimal model based on a predefined threshold, and in some embodiments, the customer may select the minimal model based on a comparison of the received minimal models. Thus, in some embodiments, for example, the customer may select the minimal model with the highest score.

[0072] The aggregator 202 can receive a selection and / or request for a minimal model to be removed and can provide the requested and / or selected minimal model to the customer environment 212. The minimal model to be removed can be provided to the customer environment via the protocol manager 304 and / or the FL connection manager 306. The customer environment 212 can receive the selected minimal model to be removed from the aggregator 202 and can store the minimal model to be removed in the model store 218. In some embodiments, the customer environment 212 can receive the minimal model to be removed via the FL connection manager 316 and / or the protocol manager 314.

[0073] At block 518, the customer environment 212 trains a fusion model. In some embodiments, the customer environment 212 can train a fusion model on the received pruned minimal models. The fusion model can be trained by the local training manager 312. In some embodiments, training the fusion model can include fusing the minimal models into the fusion model. In some embodiments, the fusion model can include, for example, a neural network such as FedAVG (e.g., Gradient Average, Fed+, FedPox), a decision tree such as ID3 fusion, reinforcement learning such as iterative average or FedAvg, a linear classifier such as iterative average, and / or a naive Bayes such as naive Bayes fusion with differential privacy.

[0074] In some embodiments, training the fusion model may include combining the binaries from each of the received minimal models, which in some embodiments may facilitate and simplify the creation of the fusion model.

[0075] At block 520, a task-specific head, specifically a task-specific classifier head, is generated and applied to the fusion model. This task-specific classifier head may be generated by the customer environment, specifically by the local training manager 312. The fusion model may be stored in the model store 218 of the customer environment in which the fusion model was trained and / or generated.

[0076] 6, a flow chart illustrating one embodiment of a process 600 for generating a fused model is shown. The process 600 may be performed by the customer environment 212. The process 600 begins at block 602, where the customer environment 212 receives and implements a global model. The global model may be received by the customer environment 212 from an aggregator. The customer environment 212 may store the global model in a model store 218 of the customer environment.

[0077] At block 604, data is collected at the customer environment 212. This data may include local data generated by or associated with the customer environment. For example, the customer environment 212 may be associated with a hospital. The local data may be some or all of the data generated by or associated with the hospital. In some embodiments, this data may be provided to or accessed by the customer environment. The local data collected at the customer environment 212 may be stored in a data store 216 for that customer environment 212 by a local data manager 318.

[0078] At block 606, a minimal model is trained by the customer environment 212. In some embodiments, the minimal model is trained on top of the global model. A minimal model trained by a customer environment 212 may be trained specifically by the local training manager 312 of that customer environment 212.

[0079] A minimal model for a customer environment 212 may be trained using local data for that customer environment 212. This local data may be obtained from the data store 216 for that customer environment 212 by a local data manager 318 for that customer environment 212. Thus, in some embodiments, training the minimal model may include obtaining local data from the data store 216 via, for example, the local data manager 318.

[0080] At block 608, the minimal model head layer is removed by the customer environment 212 and the removed minimal model, in other words, the minimal model binary, is sent and / or provided to the aggregator. In some embodiments, this may include identification and removal of the head layer from the minimal model being trained, where the head layer may include a classifier head. In some embodiments, this removed minimal model may include a binary of the minimal model, which may identify, for example, layers and / or nodes of the supplemental model and weights associated with the layers and / or nodes of the supplemental model.

[0081] After the head layer of the minimal model is removed, the minimal model to be removed may be uploaded by the customer environment to the aggregator 202, i.e., the binary of the minimal model may be uploaded. This may include the protocol manager 314 and / or the FL connection manager 316 of the customer environment 212 communicating the minimal model to be removed to the connection manager 306 and / or the protocol manager 304 of the aggregator 202. The aggregator 202 may receive the minimal model to be removed from the customer environment 212 and may store the minimal model to be removed in the global model store 206.

[0082] At block 610, the customer environment selects, fetches, requests, and / or receives one or more minimal models to be removed from the aggregator. In some embodiments, the distribution of the minimal models to be removed may be according to a push or pull mechanism. In some embodiments, for example, one or more minimal models and / or updates may be pushed by the aggregator 202 to the customer environment 212. In other embodiments, the customer environment 212 may select and / or request one or more minimal models.

[0083] In embodiments where minimal models are pushed to customer environments, the aggregator 202 can track the number of minimal models received and changes in the minimal models compared to the global models. In some embodiments, updates can be pushed by the aggregator, for example, when a predetermined period of time has passed or a threshold of changes is met.

[0084] In embodiments where the minimal models are pulled to the customer environment 212, the customer may request, via the customer environment, information characterizing the one or more minimal models to be removed. This information may include, for example, the scores of the minimal models, metadata associated with the minimal models, etc. In some embodiments, the customer may select and / or request, via the customer environment, one or more minimal models. In some embodiments, these one or more minimal models may be selected based on this information characterizing the one or more minimal models to be removed. For example, the customer may select and / or request one or more minimal models if the requested one or more minimal models have information indicative of one or more desired attributes, such as, for example, sufficiently high confidence, training of the minimal model for one or more desired tasks, etc. In some embodiments, the customer may select the minimal model based on a predefined threshold, and in some embodiments, the customer may select the minimal model based on a comparison of the received minimal models. Thus, in some embodiments, for example, the customer may select the minimal model with the highest score.

[0085] The aggregator 202 can receive a selection and / or request for a minimal model to be removed and can provide the requested and / or selected minimal model to the customer environment 212. The minimal model to be removed can be provided to the customer environment via the protocol manager 304 and / or the FL connection manager 306. The customer environment 212 can receive the selected minimal model to be removed from the aggregator 202 and can store the minimal model to be removed in the model store 218. In some embodiments, the customer environment 212 can receive the minimal model to be removed via the FL connection manager 316 and / or the protocol manager 314.

[0086] At block 612, the authenticity of the received minimal model to be removed is verified. In some embodiments, this verification of authenticity can be performed by the customer environment 212. In some embodiments, the verification of authenticity can be performed using an external trust establishment system, which can include, for example, the use of a blockchain. In some embodiments, the external trust establishment system can verify the authenticity of the received minimal model, for example, via distributed consensus.

[0087] At block 614, the received minimal model and / or updates are stored by the customer environment 212. In some embodiments, this may include storing the received minimal model and / or updates in the model store 218.

[0088] At block 616, a fusion model is constructed on the received minimal models. In some embodiments, the fusion model can be created from a combination of the supplemental models and / or the fusion model can be trained on the supplemental models. The fusion model can be trained by the local training manager 312. In some embodiments, creating the fusion model based on the received minimal models to be removed can include combining data, such as binaries of the minimal models to be removed, representing layers and weights from each of the multiple minimal models.

[0089] In some embodiments, training the fusion model may include fusing the minimal models into a fusion model. In some embodiments, the fusion model may include neural networks such as decision trees, such as FedAVG(,Gradient Average, Fed+, FedPox);ID3 fusion, reinforcement learning, such as iterative average or FedAvg, linear classifiers, such as iterative average, and / or Naive Bayes, such as Naive Bayes fusion with differential privacy.

[0090] At block 618, a task-specific head, specifically a task-specific classifier head, is generated and applied to the fusion model. This task-specific classifier head may be generated by the customer environment, specifically by the local training manager 312. The fusion model may be stored in the model store 218 of the customer environment in which the fusion model was trained and / or generated.

[0091] The fusion model is stored at block 620. The fusion model may be stored in the model store 218 of the customer environment in which it was trained and / or generated.

[0092] 7, a flow chart illustrating an embodiment of a process 700 for generating an output using a fusion model is shown. The process 700 may be executed by the customer environment 212. The process 700 begins at block 702, where data is received for use in generating an output of a machine learning model, specifically data for generating an output using a fusion model. At block 704, the data is fed into a global model. The global model may generate a number of intermediate outputs, also referred to herein as first intermediate outputs, such as one or several layer outputs. As shown at block 706, these intermediate outputs of the global model may be received and / or collected.

[0093] At block 708, the intermediate outputs of the global model are fed into the multiple minimal models from which the fused model was generated. Each of these minimal models may generate one or several intermediate outputs, also referred to herein as second intermediate outputs, which may be received and / or collected as shown at block 710.

[0094] After the intermediate outputs from the minimal model are received and / or collected, process 700 proceeds to block 712, where the intermediate outputs of the minimal model and / or the global model are populated into the fusion model. In some embodiments where the outputs of the minimal model are not desired, steps 708 and 710 may be skipped and the process may proceed directly from block 706 to block 712, where the intermediate outputs of the global model are populated into the fusion model. In some embodiments, in addition to populating the intermediate outputs into the fusion model, some or all of the data received at block 702 may be populated into the fusion model.

[0095] At block 714, the classifier head of the fusion model generates outputs and / or makes predictions based on the inputs received from the layers of the fusion model, based on the inputs fed into the fusion model, which may be output by the fusion model.

[0096] Example implementation FIG. 8 is a block diagram 800 illustrating an example IaaS architecture pattern, according to at least one embodiment. A service operator 802 can be communicatively coupled to a secure host tenancy 804, which can include a virtual cloud network (VCN) 806 and a secure host subnet 808. In some examples, the service operator 802 can use one or more client computing devices, which can be portable handheld devices (e.g., iPhone, mobile phone, iPad, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass head mounted display), running software such as Microsoft Windows Mobile, and / or various mobile operating systems such as iOS, WindowsPhone, Android, BlackBerry8, PalmOS, and Internet, email, short message service (SMS), Blackberry, or other communication protocols in effect. Alternatively, the client computing devices can be general purpose personal computers, including, for example, personal computers and / or laptop computers running various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems. The client computing device may be a workstation computer running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS.Alternatively, or in addition, the client computing device may be any other electronic device, such as a thin-client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without a Kinect® gesture input device), and / or a personal messaging device capable of communicating over a network with access to the VCN 806 and / or the Internet.

[0097] The VCN 806 may include a local peering gateway (LPG) 810 that may be communicatively coupled to a secure shell (SSH) VCN 812 via an LPG 810 that is included in the SSH VCN 812. The SSH VCN 812 may include an SSH subnet 814, which may be communicatively coupled to a control plane VCN 816 via an LPG 810 that is included in the control plane VCN 816. The SSH VCN 812 may also be communicatively coupled to a data plane VCN 818 via the LPG 810. The control plane VCN 816 and the data plane VCN 818 may be included in a service tenancy 819, which may be owned and / or operated by the IaaS provider.

[0098] The control plane VCN 816 may include a control plane demilitarized zone (DMZ) tier 820 that functions as a perimeter network (e.g., a portion of an enterprise network between the enterprise intranet and an external network). DMZ-based servers have limited responsibility and may help prevent security breaches. Additionally, the DMZ tier 820 may include one or more load balancer (LB) subnets 822, a control plane app tier 824 that may include an app subnet 826, and a control plane data tier 828 that may include a database (DB) subnet 830 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 822 included in the control plane DMZ tier 820 may be communicatively coupled to the app subnet 826 included in the control plane app tier 824 and an Internet gateway 834 that may be included in the control plane VCN 816, and the app subnet 826 may be communicatively coupled to the DB subnet 830 included in the control plane data tier 828, as well as a service gateway 836 and a network address translation (NAT) gateway 838. The control plane VCN 816 may include a service gateway 836 and a NAT gateway 838 .

[0099] The control plane VCN 816 can include a data plane mirrored app layer 840 that can include an app subnet 826. The app subnet 826 included in the data plane mirrored app layer 840 can include a virtual network interface controller (VNIC) 842 on which a compute instance 844 can run. The compute instance 844 can communicatively couple the app subnet 826 of the data plane mirrored app layer 840 to the app subnet 826 that can be included in the data plane app layer 846.

[0100] The data plane VCN 818 can include a data plane app layer 846, a data plane DMZ layer 848, and a data plane data layer 850. The data plane DMZ layer 848 can include a LB subnet 822, which can be communicatively coupled to an app subnet 826 of the data plane app layer 846 and an Internet gateway 834 of the data plane VCN 818. The app subnet 826 can be communicatively coupled to a service gateway 836 of the data plane VCN 818 and a NAT gateway 838 of the data plane VCN 818. The data plane data layer 850 can also include a DB subnet 830, which can be communicatively coupled to the app subnet 826 of the data plane app layer 846.

[0101] The Internet gateways 834 of the control plane VCNs 816 and data plane VCNs 818 may be communicatively coupled to a metadata management service 852, which may be communicatively coupled to the public Internet 854. The public Internet 854 may be communicatively coupled to a NAT gateway 838 of the control plane VCNs 816 and data plane VCNs 818. The service gateways 836 of the control plane VCNs 816 and data plane VCNs 818 may be communicatively coupled to cloud services 856.

[0102] In some examples, a service gateway 836 in the control plane VCN 816 or the data plan VCN 818 can make application programming interface (API) calls to cloud services 856 without traversing the public Internet 854. API calls from the service gateway 836 to the cloud services 856 can be one-way: the service gateway 836 can make an API call to the cloud services 856, and the cloud services 856 can send the requested data to the service gateway 836. However, the cloud services 856 may not be able to initiate the API call to the service gateway 836.

[0103] In some examples, secure host tenancy 804 can be directly connected to service tenancy 819 or can be otherwise separate. Secure host subnet 808 can communicate with SSH subnet 814 through LPG 810, which can allow bidirectional communication through otherwise separate systems. Connecting secure host subnet 808 to SSH subnet 814 can give secure host subnet 808 access to other entities in service tenancy 819.

[0104] The control plane VCN 816 can enable users of the service tenancy 819 to set up or provision desired resources. The desired resources provisioned in the control plane VCN 816 can be deployed or used in the data plane VCN 818. In some examples, the control plane VCN 816 can be separate from the data plane VCN 818, and the data plane mirror app layer 840 of the control plane VCN 816 can communicate with the data plane app layer 846 of the data plane VCN 818 via a VNIC 842, which can be included in the data plane mirror app layer 840 and the data plane app layer 846.

[0105] In some examples, a user or customer of the system may make a request, such as, for example, a create, read, update, or delete (CRUD) operation, over the public internet 854, which may communicate the request to a metadata management service 852. The metadata management service 852 may communicate the request to the control plane VCN 816 over the internet gateway 834. The request may be received by a LB subnet 822 included in the control plane DMZ layer 820. The LB subnet 822 may determine that the request is valid, and in response to this determination, the LB subnet 822 may send the request to an app subnet 826 included in the control plane app layer 824. If the request is validated and a call to the public internet 854 is required, the call to the public internet 854 may be sent to a NAT gateway 838, which may make the call to the public internet 854. Memory that may be desirable to be stored with the request may be stored in the DB subnet 830.

[0106] In some examples, the data plane mirror app layer 840 can facilitate direct communication between the control plane VCN 816 and the data plane VCN 818. For example, it may be desirable to apply configuration changes, updates, or other suitable modifications to resources included in the data plane VCN 818. Through the VNIC 842, the control plane VCN 816 can communicate directly with the resources included in the data plane VCN 818, thereby enabling it to perform configuration changes, updates, or other suitable modifications to the resources included in the data plane VCN 818.

[0107] In some embodiments, the control plane VCN 816 and the data plane VCN 818 can be included in the service tenancy 819. In this case, a user or customer of the system may not own or operate either the control plane VCN 816 or the data plane VCN 818. Instead, an IaaS provider may own or operate the control plane VCN 816 and the data plane VCN 818, both of which may be included in the service tenancy 819. This embodiment may allow for network isolation that may prevent a user or customer from interacting with the resources of other users or other customers. This embodiment also allows a user or customer of the system to store databases privately without having to rely on the public Internet 854, which may not have the desired level of security for storage.

[0108] In another embodiment, the LB subnet 822 included in the control plane VCN 816 may be configured to receive signals from the service gateway 836. In this embodiment, the control plane VCN 816 and the data plane VCN 818 may be configured to be called by the IaaS provider's customers without calling the public Internet 854. The IaaS provider's customers may desire this embodiment because databases used by the customers may be stored in the service tenancy 819, which may be controlled by the IaaS provider and isolated from the public Internet 854.

[0109] 9 is a block diagram 900 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 902 (e.g., service operator 802 of FIG. 8 ) can be communicatively coupled to a secure host tenancy 904 (e.g., secure host tenancy 804 of FIG. 8 ), which can include a virtual cloud network (VCN) 906 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 908 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 906 can include a local peering gateway (LPG) 910 (e.g., LPG 810 of FIG. 8 ), which can be communicatively coupled to a secure shell (SSH) VCN 912 (e.g., SSH VCN 812 of FIG. 8 ) via the LPG 810 included in the SSH VCN 912. The SSH VCN 912 can include an SSH subnet 914 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively coupled to a control plane VCN 916 via an LPG 910 that is included in the control plane VCN 916 (e.g., control plane VCN 816 in FIG. 8 ). The control plane VCN 916 can be included in a service tenancy 919 (e.g., service tenancy 819 in FIG. 8 ), and the data plane VCN 918 (e.g., data plane VCN 818 in FIG. 8 ) can be included in a customer tenancy 921, which can be owned or operated by a user or customer of the system.

[0110] The control plane VCN 916 may include a control plane DMZ layer 920 (e.g., control plane DMZ layer 820 of FIG. 8 ) that may include a LB subnet 922 (e.g., LB subnet 822 of FIG. 8 ), a control plane app layer 924 (e.g., control plane app layer 824 of FIG. 8 ) that may include an app subnet 926 (e.g., app subnet 826 of FIG. 8 ), and a control plane data layer 928 (e.g., control plane data layer 828 of FIG. 8 ) that may include a database (DB) subnet 930 (e.g., similar to DB subnet 830 of FIG. 8 ). The LB subnet 922 included in the control plane DMZ tier 920 may be communicatively coupled to an app subnet 926 included in the control plane app tier 924 and to an Internet gateway 934 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 916, and the app subnet 926 can be communicatively coupled to a DB subnet 930, a service gateway 936 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 938 (e.g., NAT gateway 838 in FIG. 8 ) included in the control plane data tier 928. The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0111] The control plane VCN 916 can include a data plane mirror app layer 940 (e.g., data plane mirror app layer 840 of FIG. 8 ), which can include an app subnet 926. The app subnet 926 included in the data plane mirror app layer 940 can include a virtual network interface controller (VNIC) 942 (e.g., VNIC of 842) on which a compute instance 944 (e.g., similar to compute instance 844 of FIG. 8 ) can run. The compute instance 944 can facilitate communication between the app subnet 926 of the data plane mirror app layer 940 and the app subnet 926 that can be included in the data plane app layer 946 (e.g., data plane app layer 846 of FIG. 8 ) via the VNIC 942 included in the data plane mirror app layer 940 and the VNIC 942 included in the data plane app layer 946.

[0112] An internet gateway 934 included in the control plane VCN 916 can be communicatively coupled to a metadata management service 952 (e.g., metadata management service 852 in FIG. 8), which can be communicatively coupled to a public internet 954 (e.g., public internet 854 in FIG. 8). The public internet 954 can be communicatively coupled to a NAT gateway 938 included in the control plane VCN 916. A service gateway 936 included in the control plane VCN 916 can be communicatively coupled to cloud services 956 (e.g., cloud services 856 in FIG. 8).

[0113] In some examples, the data plane VCN 918 can be included in the customer tenancy 921. In this case, the IaaS provider can provide a control plane VCN 916 for each customer, and the IaaS provider can set up a unique compute instance 944 included in the service tenancy 919 for each customer. Each compute instance 944 can enable communication between the control plane VCN 916 included in the service tenancy 919 and the data plane VCN 918 included in the customer tenancy 921. The compute instance 944 can enable resources provisioned in the control plane VCN 916 included in the service tenancy 919 to be deployed or otherwise used in the data plane VCN 918 included in the customer tenancy 921.

[0114] In another example, a customer of the IaaS provider may have a database that resides in customer tenancy 921. In this example, control plane VCN 916 may include data plane mirror app tier 940, which may include app subnet 926. Data plane mirror app tier 940 may reside in data plane VCN 918, but data plane mirror app tier 940 may not reside in data plane VCN 918. That is, data plane mirror app tier 940 may have access to customer tenancy 921, but data plane mirror app tier 940 may not reside in data plane VCN 918 or be owned or operated by the IaaS provider customer. Data plane mirror app tier 940 may be configured to make calls to data plane VCN 918, but may not be configured to make calls to any entities included in control plane VCN 916. A customer may desire to deploy or use resources in data plane VCN 918 that are provisioned in control plane VCN 916, and data plane mirror app tier 940 may facilitate the customer's desired deployment or other use of the resources.

[0115] In some embodiments, the IaaS provider's customer can apply filters to the data plane VCN 918. In this embodiment, the customer can determine what the data plane VCN 918 can access, and the customer can limit access from the data plane VCN 918 to the public internet 954. The IaaS provider may not be able to apply filters or control the data plane VCN 918's access to external networks or databases. Applying customer filters and controls to the data plane VCN 918 contained in the customer tenancy 921 can help isolate the data plane VCN 918 from other customers and the public internet 954.

[0116] In some embodiments, cloud services 956 can be called by the service gateway 936 to access services that may not be on the public internet 954, on the control plane VCN 916, or on the data plane VCN 918. The connection between the cloud services 956 and the control plane VCN 916 or the data plane VCN 918 may not be present or continuous. The cloud services 956 may be on another network owned or operated by the IaaS provider. The cloud services 956 may be configured to receive calls from the service gateway 936 or may be configured not to receive calls from the public internet 954. Some cloud services 956 may be isolated from other cloud services 956, and the control plane VCN 916 may be isolated from cloud services 956 that may not be in the same region as the control plane VCN 916. For example, the control plane VCN 916 may be located in “Region 1” and cloud service “Deployment 11” may be located in Region 1 and Region 2. If a call to deployment 11 is made by a service gateway 936 included in a control plane VCN 916 in region 1, the call may be sent to deployment 11 in region 1. In this example, control plane VCN 916, or deployment 11 in region 1, may not be communicatively coupled to or in communication with deployment 11 in region 2.

[0117] 10 is a block diagram 1000 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1002 (e.g., service operator 802 of FIG. 8 ) can be communicatively coupled to a secure host tenancy 1004 (e.g., secure host tenancy 804 of FIG. 8 ), which can include a virtual cloud network (VCN) 1006 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 1008 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 1006 can include an LPG 1010 (e.g., LPG 810 of FIG. 8 ) that can be communicatively coupled to an SSH VCN 1011 (e.g., SSH VCN 812 of FIG. 8 ) via an LPG 1010 included in the SSH VCN 1012. The SSH VCN 1012 can include an SSH subnet 1014 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively coupled to a control plane VCN 1016 via an LPG 1010 included in the control plane VCN 1016 (e.g., control plane VCN 816 in FIG. 8 ) and to a data plane VCN 1018 via an LPG 1010 included in the data plane VCN 1018 (e.g., data plane 818 in FIG. 8 ). The control plane VCN 1016 and the data plane VCN 1018 can be included in a service tenancy 1019 (e.g., service tenancy 819 in FIG. 8 ).

[0118] The control plane VCN 1016 may include a control plane DMZ tier 1020 (e.g., control plane DMZ tier 820 of FIG. 8 ) that may include a load balancer (LB) subnet 1022 (e.g., LB subnet 822 of FIG. 8 ), a control plane app tier 1024 (e.g., control plane app tier 824 of FIG. 8 ) that may include an app subnet 1026 (e.g., similar to app subnet 826 of FIG. 8 ), and a control plane data tier 1028 (e.g., control plane data tier 828 of FIG. 8 ) that may include a DB subnet 1030. The LB subnet 1022 included in the control plane DMZ layer 1020 can be communicatively coupled to an app subnet 1026 included in the control plane app layer 1024 and can be communicatively coupled to an Internet gateway 1034 (e.g., Internet gateway 834 of FIG. 8 ) that can be included in the control plane VCN 1016, and the app subnet 1026 can be communicatively coupled to a DB subnet 1030 included in the control plane data layer 1028 and can be communicatively coupled to a service gateway 1036 (e.g., service gateway of FIG. 8 ) and a network address translation (NAT) gateway 1038 (e.g., NAT gateway 838 of FIG. 8 ). The control plane VCN 1016 can include the service gateway 1036 and the NAT gateway 1038.

[0119] The data plane VCN 1018 may include a data plane app layer 1046 (e.g., data plane app layer 846 of FIG. 8 ), a data plane DMZ layer 1048 (e.g., data plane DMZ layer 848 of FIG. 8 ), and a data plane data layer 1050 (e.g., data plane data layer 850 of FIG. 8 ). The data plane DMZ layer 1048 may include a LB subnet 1022, which may be communicatively coupled to a trusted app subnet 1060, which may be communicatively coupled to an untrusted app subnet 1062 of the data plane app layer 1046 and an Internet gateway 1034 included in the data plane VCN 1018. The trusted app subnet 1060 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018, which may be communicatively coupled to a NAT gateway 1038 included in the data plane VCN 1018, and a DB subnet 1030 included in the data plane data layer 1050. The untrusted app subnet 1062 can be communicatively connected to a service gateway 1036 included in the data plane VCN 1018 and a DB subnet 1030 included in the data plane data layer 1050. The data plane data layer 1050 can include a DB subnet 1030 that can be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018.

[0120] The untrusted app subnet 1062 may include one or more primary VNICs 1064(1)-(N) that may be communicatively coupled to tenancy virtual machines (VMs) 1066(1)-(N). Each tenancy VM 1066(1)-(N) may be communicatively coupled to a respective app subnet 1067(1)-(N) that may be included in a respective container egress VCN 1068(1)-(N) that may be included in a respective customer tenancy 1070(1)-(N). Each secondary VNIC 1072(1)-(N) may facilitate communication between the untrusted app subnet 1062 included in the data plane VCN 1018 and the app subnet included in the container egress VCN 1068(1)-(N). Each container egress VCN 1068(1)-(N) may include a NAT gateway 1038 that may be communicatively coupled to the public internet 1054 (e.g., public internet 854 of FIG. 8).

[0121] An Internet gateway 1034 included in the control plane VCN 1016 and included in the data plane VCN 1018 can be communicatively coupled to a metadata management service 1052 (e.g., metadata management system 852 of FIG. 8 ), which can be communicatively coupled to the public Internet 1054. The public Internet 1054 can be communicatively coupled to a NAT gateway 1038 included in the control plane VCN 1016 and included in the data plane VCN 1018. A service gateway 1036 included in the control plane VCN 1016 and included in the data plane VCN 1018 can be communicatively coupled to cloud services 1056.

[0122] In some embodiments, the data plane VCN 1018 may be integrated with a customer tenancy 1070. This integration may be beneficial or desirable for the IaaS provider's customer, such as when support at code execution is desired. The customer may provide for the execution of code that may be disruptive, may communicate with other customer resources, or may cause other undesirable effects. In response, the IaaS provider can decide whether to run code provided to the IaaS provider by the customer.

[0123] In some examples, a customer of an IaaS provider may grant temporary network access to the IaaS provider and request functionality to be added to the data plane layer app 1046. The code to perform the functionality may be executed in the VMs 1066(1)-(N), and the code may not be configured to run elsewhere on the data plane VCN 1018. Each VM 1066(1)-(N) may be connected to one customer tenancy 1070. Each container 1071(1)-(N) contained in a VM 1066(1)-(N) may be configured to execute code. In this case, there may be double isolation (e.g., code execution in container 1071(1)-(N), container 1071(1)-(N) may be contained in at least VMs 1066(1)-(N) contained in the untrusted app subnet 1062), which may help prevent erroneous or unwanted code from damaging the IaaS provider's network or damaging another customer's network. Containers 1071(1)-(N) may be communicatively coupled to customer tenancy 1070 and may be configured to send or receive data from customer tenancy 1070. Containers 1071(1)-(N) may not be configured to send or receive data from any other entity in data plane VCN 1018. Once code execution is complete, the IaaS provider may kill or otherwise discard containers 1071(1)-(N).

[0124] In some embodiments, the trusted app subnet 1060 may execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted app subnet 1060 may be communicatively coupled to the DB subnet 1030 and configured to perform CRUD operations on the DB subnet 1030. The untrusted app subnet 1062 may be communicatively coupled to the DB subnet 1030, but in this embodiment, the untrusted app subnet may be configured to perform read operations on the DB subnet 1030. Containers 1071(1)-(N) that may be included in each customer's VMs 1066(1)-(N) and that may execute code from the customer may not be communicatively coupled to the DB subnet 1030.

[0125] In other embodiments, the control plane VCN 1016 and the data plane VCN 1018 may not be directly communicatively coupled. In this embodiment, there may not be direct communication between the control plane VCN 1016 and the data plane VCN 1018. However, communication may occur indirectly through at least one method. The LPG 1010 may be established by an IaaS provider that may facilitate communication between the control plane VCN 1016 and the data plane VCN 1018. In another example, the control plane VCN 1016 or the data plane VCN 1018 may make a call to a cloud service 1056 through a service gateway 1036. For example, a call from the control plane VCN 1016 to the cloud service 1056 may include a request for a service that may communicate with the data plane VCN 1018.

[0126] 11 is a block diagram 1100 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1102 (e.g., service operator 802 of FIG. 8 ) can be communicatively coupled to a secure host tenancy 1104 (e.g., secure host tenancy 804 of FIG. 8 ) which can include a virtual cloud network (VCN) 1106 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 1108 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 1106 can include an LPG 1110 (e.g., LPG 810 of FIG. 8 ) which can be communicatively coupled to an SSH VCN 1112 (e.g., SSH VCN 812 of FIG. 8 ) via an LPG 1110 included in the SSH VCN 1112. SSH VCN 1112 can include an SSH subnet 1114 (e.g., SSH subnet 814 in FIG. 8 ), which can be communicatively coupled to a control plane VCN 1116 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 1110 included in control plane VCN 1116 and a data plane VCN 1118 (e.g., data plane 818 in FIG. 8 ) via an LPG 1110 included in data plane VCN 1118. The control plane VCN 1116 and the data plane VCN 1118 can be included in a service tenancy 1119 (e.g., service tenancy 819 in FIG. 8 ).

[0127] The control plane VCN 1116 may include a control plane DMZ layer 1120 (e.g., control plane DMZ layer 820 of FIG. 8 ) which may include a LB subnet 1122 (e.g., LB subnet 822 of FIG. 8 ), a control plane app layer 1124 (e.g., control plane app layer 824 of FIG. 8 ) which may include an app subnet 1126 (e.g., app subnet 826 of FIG. 8 ), and a control plane data layer 1128 (e.g., control plane data layer 828 of FIG. 8 ) which may include a DB subnet 1130 (e.g., DB subnet 1030 of FIG. 10 ). The LB subnet 1122 included in the control plane DMZ tier 1120 can be communicatively coupled to an app subnet 1126 included in the control plane app tier 1124 and an Internet gateway 1134 (e.g., Internet gateway 834 in FIG. 8 ) that can be included in the control plane VCN 1116, and the app subnet 1126 can be communicatively coupled to a DB subnet 1130 and a service gateway 1136 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 1138 (e.g., NAT gateway 838 in FIG. 8 ) included in the control plane data tier 1128. The control plane VCN 1116 can include the service gateway 1136 and the NAT gateway 1138.

[0128] The data plane VCN 1118 can include a data plane app layer 1146 (e.g., data plane app layer 846 in FIG. 8), a data plane DMZ layer 1148 (e.g., data plane DMZ layer 848 in FIG. 8), and a data plane data layer 1150 (e.g., data plane data layer 850 in FIG. 8). The data plane DMZ layer 1148 can include a trusted app subnet 1160 (e.g., trusted app subnet 1060 in FIG. 10) and a LB subnet 1122 that can be communicatively coupled to the data plane app layer 1146 and an untrusted app subnet 1162 (e.g., untrusted app subnet 1062 in FIG. 10) of the Internet gateway 1134 included in the data plane VCN 1118. The trusted app subnet 1160 can be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118 and a NAT gateway 1138 included in the data plane VCN 1118, and a DB subnet 1130 included in the data plane data layer 1150. The untrusted app subnet 1162 can be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118 and a DB subnet 1130 included in the data plane data layer 1150. The data plane data layer 1150 can include a DB subnet 1130 that can be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118.

[0129] The untrusted app subnet 1162 may include primary VNICs 1164(1)-(N) that may be communicatively coupled to tenancy virtual machines (VMs) 1166(1)-(N) that reside within the untrusted app subnet 1162. Each tenancy VM 1166(1)-(N) may execute code within a respective container 1167(1)-(N) and may be communicatively coupled to an app subnet 1126 that may be included in a data plane app layer 1146 that may be included in a container egress VCN 1168. Each secondary VNIC 1172(1)-(N) may facilitate communication between the untrusted app subnet 1162 included in the data plane VCN 1118 and the app subnet included in the container egress VCN 1168. The container egress VCN may include a NAT gateway 1138 that may be communicatively coupled to the public Internet 1154 (e.g., public Internet 854 of FIG. 8 ).

[0130] The Internet gateway 1134 included in the control plane VCN 1116 and included in the data plane VCN 1118 can be communicatively coupled to a metadata management service 1152 (e.g., metadata management system 852 of FIG. 8 ), which can be communicatively coupled to the public Internet 1154. The public Internet 1154 can be communicatively coupled to a NAT gateway 1138 included in the control plane VCN 1116 and included in the data plane VCN 1118. The service gateway 1136 included in the control plane VCN 1116 and included in the data plane VCN 1118 can be communicatively coupled to cloud services 1156.

[0131] In some examples, the pattern illustrated by the architecture of block diagram 1100 of FIG. 11 may be an exception to the pattern illustrated by the architecture of block diagram 1000 of FIG. 10 and may be desirable for an IaaS provider's customer when the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected area). Each container 1167(1)-(N) contained in each customer's VM 1166(1)-(N) is accessible in real time by the customer. The containers 1167(1)-(N) may be configured to make calls to a respective secondary VNIC 1172(1)-(N) contained in the app subnet 1126 of the data plane app tier 1146 that may be contained in a container egress VCN 1168. The secondary VNIC 1172(1)-(N) may send the call to a NAT gateway 1138 that may send the call to the public Internet 1154. In this example, containers 1167(1)-(N) that a customer can access in real time can be isolated from control plane VCN 1116 and can be isolated from other entities included in data plane VCN 1118. Containers 1167(1)-(N) may be isolated from resources from other customers.

[0132] In another example, a customer can use container 1167(1)-(N) to invoke cloud service 1156. In this example, the customer can execute code within container 1167(1)-(N) that requests a service from cloud service 1156. Container 1167(1)-(N) can send the request to secondary VNIC 1172(1)-(N), which can send the request to a NAT gateway that can send the request to public Internet 1154. Public Internet 1154 can send the request via Internet gateway 1134 to LB subnet 1122 included in control plane VCN 1116. In response to determining that the request is valid, the LB subnet can send the request to app subnet 1126, which can send the request to cloud service 1156 via service gateway 1136.

[0133] It should be understood that the IaaS architectures 800, 900, 1000, 1100 depicted in the figures may have components other than those depicted. Additionally, the embodiments depicted in the figures are only some examples of cloud infrastructure systems that may incorporate embodiments of the present disclosure. In some other embodiments, the IaaS systems may have more or fewer components than depicted, may combine two or more components, or may have a different configuration or arrangement of components.

[0134] In certain embodiments, the IaaS system described herein may include a suite of application, middleware, and database services products that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such an IaaS system is Oracle Cloud Infrastructure (OCI), offered by the present assignee.

[0135] 12 illustrates an exemplary computer system 1200 on which various embodiments of the disclosure may be implemented. The system 1200 may be used to implement any of the computer systems described above. As shown in the figure, the computer system 1200 includes a processing unit 1204 that communicates with a number of peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing accelerator 1206, an I / O subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. The storage subsystem 1218 includes a tangible computer-readable storage medium 1222 and a system memory 1210.

[0136] Bus subsystem 1202 provides a mechanism that allows the various components and subsystems of computer system 1200 to communicate with each other as intended. Although bus subsystem 1202 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1202 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. It may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0137] The processing unit 1204 may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers) and controls the operation of the computer system 1200. One or more processors may be included in the processing unit 1204. These processors may include single-core processors or multi-core processors. In particular embodiments, the processing unit 1204 may be implemented as one or more independent processing units 1232 and / or 1234 with a single-core processor or a multi-core processor included in each processing unit. In other embodiments, the processing unit 1204 may be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0138] In various embodiments, the processing unit 1204 may execute various programs in response to program code and may maintain multiple simultaneously executing programs or processes. At any time, some or all of the program code being executed may reside in the processor 1204 and / or in the memory subsystem 1218. Through appropriate programming, the processor 1204 may provide the various functions discussed above. The computer system 1200 may further include a processing accelerator 1206, which may include a digital signal processor (DSP), a special purpose processor, or the like.

[0139] The I / O subsystem 1208 can include user interface input devices and user interface output devices. User interface input devices can include keyboards, pointing devices such as mice or trackballs, touch pads or touch screens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices can include, for example, motion sensing and / or gesture recognition devices such as Microsoft Kinect® motion sensors, allowing users to control and interact with input devices such as Microsoft Xbox® 360 game controllers through a natural user interface using gestures and voice commands. User interface input devices can also include eye gesture recognition devices such as the Google Glass® blink detector that detects eye activity from a user (e.g., “blinking” while taking a picture and / or selecting a menu) and translates eye gestures as input to an input device (e.g., Google Glass®). Additionally, the user interface input devices may include voice recognition sensing devices that allow a user to interact with a voice recognition system (e.g., the Siri® navigator) through voice commands.

[0140] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye-tracking devices. Additionally, user interface input devices may include medical imaging input devices such as, for example, computed tomography, magnetic resonance imaging, position emission tomography, and medical ultrasound machines. User interface input devices may also include audio input devices such as, for example, MIDI keyboards, digital musical instruments, and the like.

[0141] User interface output devices may include non-visual displays such as a display subsystem, indicator lights, or audio output devices. The display subsystem may be a flat panel device such as one using a cathode ray tube (CRT), liquid crystal display (LCD) or plasma display, a projection device, a touch screen, etc. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1200 to a user or to another computer. For example, user interface output devices include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, voice output devices, and modems.

[0142] Computer system 1200 may include a storage subsystem 1218 that comprises software elements shown as currently located in system memory 1210. System memory 1210 may store program instructions that are loadable and executable on processing unit 1204, as well as data generated during the execution of these programs.

[0143] Depending on the configuration and type of computer system 1200, the system memory 1210 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or currently being operated on and executed by the processing unit 1204. In some implementations, the system memory 1210 may include a number of different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 1200, such as during start-up, may typically be stored in ROM. By way of example and not limitation, the system memory 1210 also illustrates application programs 1212, program data 1214, and an operating system 1216, which may include client applications, a web browser, a mid-tier application, a relational database management system (RDBMS), and the like. By way of example, operating systems 1216 may include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 15 OS, and Palm® OS operating systems.

[0144] The storage subsystem 1218 may also provide a tangible computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processor, provide the functionality described above may be stored in the storage subsystem 1218. These software modules or instructions may be executed by the processing unit 1204. The storage subsystem 1218 may also provide a repository for storing data used in accordance with the present disclosure.

[0145] Storage subsystem 1200 may also include a computer readable storage medium reader 1220 that can be further connected to a computer readable storage medium 1222. Together, and optionally in combination with system memory 1210, computer readable storage medium 1222 can comprehensively represent remote, local, fixed, and / or removable storage devices, as well as storage media for containing, storing, transmitting, and retrieving computer readable information on a temporary and / or more permanent basis.

[0146] The computer readable storage medium 1222 containing the code or portions of code may include any suitable medium known or used in the art, including, but not limited to, storage media and communication media, such as volatile and non-volatile, removable and non-removable media implemented in any manner or technology for storing and / or transmitting information. This may include tangible computer readable storage media, such as RAM, ROM, Electronically Erasable Programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage, or other tangible computer readable medium. This may also include intangible computer readable media, such as a data signal, data transmission, or any other medium that can be used to transmit the desired information and that can be accessed by the computing system 1200.

[0147] As an example, the computer readable storage medium 1222 may include a hard disk drive that reads or writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads or writes to a removable non-volatile magnetic disk, and an optical disk drive that reads or writes to a removable non-volatile optical disk, such as a CD-ROM, DVD, Blu-Ray® disk, or other optical medium. The computer readable storage medium 1222 may include, but is not limited to, a Zip® drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, a digital video tape, and the like. The computer readable storage medium 1222 may also include a solid state drive (SSD) based on non-volatile memory, such as a flash memory-based SSD, an enterprise flash drive, an SSD based on volatile memory, such as solid state RAM, dynamic RAM, static RAM, such as solid state ROM, a DRAM-based SSD, a magnetoresistive RAM (MRAM) SSD, and a hybrid SSD that uses a combination of DRAM and a flash memory-based SSD. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 1200.

[0148] The communication subsystem 1224 provides an interface to other computer systems and networks. The communication subsystem 1224 serves as an interface for transmitting and receiving data from the computer system 1200 to and from other systems. For example, the communication subsystem 1224 may enable the computer system 1200 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1224 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., using cellular technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution)), WiFi (IEEE 802.11 family standard, or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components. In some embodiments, the communication subsystem 1224 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0149] In some embodiments, the communications subsystem 1224 may also receive incoming communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc., on behalf of one or more users who may use the computer system 1200.

[0150] As an example, the communications subsystem 1224 may be configured to receive data feeds 1226 in real time from users of social networks and / or other communications services, such as web feeds, such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third party information sources.

[0151] Additionally, the communications subsystem 1224 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1228 of real-time events and / or event updates 1230, which may be continuous or essentially unlimited with no apparent end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like.

[0152] The communications subsystem 1224 may also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc. to one or more databases that may be in communication with one or more streaming data source computers coupled to the computer system 1200.

[0153] The computer system 1200 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0154] Due to the ever-changing nature of computers and networks, the description of the computer system 1200 shown in the figure is intended as a specific example only. Many other configurations are possible, with more or fewer components than the system shown in the figure. For example, customized hardware may also be used, or particular elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices, such as network input / output devices, may be used. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other approaches and / or methods for implementing various embodiments.

Claims

1. receiving a global model, the global model including a plurality of layers, from a central aggregator communicatively coupled to a plurality of user environments; training a minimal model on the global model using data collected in the user environment; uploading at least a portion of the minimal model to the central aggregator; receiving a plurality of minimal models; and creating a fusion model based on the received plurality of minimal models.

2. The method of claim 1 , wherein the multiple user environments include multiple tenancies within a cloud computing network.

3. The method of claim 1 , wherein the global model comprises a deep learning model.

4. The method of claim 3 , wherein the deep learning model includes a Transformer.

5. The method of claim 3 , wherein the deep learning model comprises a Bidirectional Encoding Representation with Transformers (BERT) model.

6. The method of claim 1 , wherein the minimal model is configured to capture an output of the global model.

7. The method of claim 6 , wherein the outputs of the global model include layer outputs of at least some of the layers of the global model.

8. The method of claim 1 , wherein uploading the minimal model to the central aggregator includes removing a classifier head from the minimal model.

9. The method of claim 8 , wherein uploading the minimal model to the central aggregator comprises uploading a binary of the minimal model.

10. The method of claim 1 , wherein the plurality of minimal models is received from the central aggregator.

11. The method of claim 1 , wherein creating the fusion model based on the received plurality of minimal models comprises training the fusion model on the received minimal models.

12. The method of claim 1 , wherein creating the fusion model based on the received plurality of minimal models comprises combining data representing layers and weights from each of the plurality of minimal models.

13. 2. The method of claim 1, wherein creating the fused model based on the received plurality of minimal models comprises generating a classifier head and applying the classifier head to a layer of the fused model.

14. Receiving data for use in generating a machine learning model output; populating the data using the global model in the user environment; collecting intermediate outputs of the global model; populating the intermediate output using the fusion model; outputting a prediction using the fused model; The method of claim 1 further comprising:

15. Receiving data for use in generating a machine learning model output; populating the data using the global model in the user environment; collecting a first intermediate output of the global model; populating the first intermediate output of the global model with at least some of the plurality of minimal models; collecting second intermediate outputs of at least some of the plurality of minimal models; populating the intermediate output using the fusion model; outputting a prediction using the fusion model; and The method of claim 1 further comprising:

16. Memory, A system comprising at least one processor configured to perform the method according to any preceding claim.

17. A program for causing one or more processors to execute the method according to any one of claims 1 to 15.

Images