Utilizing microservice containers to provide tenant isolation in a multi-tenant API gateway.

Abstract

To provide a method of isolating tenants in a multi-tenant application programming interface (API) gateway.SOLUTION: A method hosts APIs for a plurality of different tenants and receives requests from many different client devices. As the requests are received, an associated tenant can be identified, and a router determines if a container instance is available to service the request. The container instance may be an empty container instance including an internal endpoint, a Web server, and a runtime environment. The empty container instance cannot be associated with a particular tenant. To associate the container instance with a tenant, a data store, such as a key-value data store, retrieves configuration files that turn the agnostic container instance into a container instance that is associated with the particular tenant and includes a configuration code to perform the requisite API functions.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 62 / 566,422, filed Sep. 30, 2017; U.S. Provisional Patent Application No. 62 / 566,431, filed Sep. 30, 2017; and U.S. Provisional Patent Application No. 62 / 566,435, filed Sep. 30, 2017. The entire disclosures of U.S. Provisional Patent Application No. 62 / 566,422, No. 62 / 566,431, and No. 62 / 566,435 are hereby incorporated by reference herein, including the appendices.

Background Art

[0002] Background An application programming interface (API) gateway is a well - managed service that enables developers to create, publish, maintain, monitor, and secure APIs for public consumption. A tenant can define APIs and expose them to these public API gateways, such as a cloud platform, which can then be made available to client devices. In large - scale API gateways, the environment may include a multi - tenant environment where multiple different tenants host APIs that are made available through the same hosted service.

Summary of the Invention

Means for Solving the Problems

[0003] Summary In some embodiments, a method for isolating tenants using containers to serve requests in a multi-tenant environment may include the steps of: receiving a first request for a first service provided by a first tenant; selecting an empty container in the multi-tenant environment; loading a first configuration that realizes the first service into the container; servicing the first request from the container; receiving a second request for a second service provided by a second tenant; flushing the first configuration from the container; and servicing the second request from the container.

[0004] In some embodiments, a non-transient computer-readable medium includes instructions that, when executed by one or more processors, cause one or more processors to perform an operation, the operation including receiving a first request for a first service provided by a first tenant, selecting an empty container in a multi-tenant environment, loading a first configuration to realize the first service into the container, servicing the first request from the container, receiving a second request for a second service provided by a second tenant, flushing the first configuration from the container, and servicing the second request from the container.

[0005] In some embodiments, the system may include one or more processors and one or more memory devices, the one or more memory devices including instructions that, when executed by one or more processors, cause one or more processors to perform an operation, the operation being to receive a first request for a first service provided by a first tenant, to select an empty container in a multitenant environment, to load a first configuration that realizes the first service into the container, to serve the first request from the container, and This includes receiving a second request for a second service provided by tenant 2, flushing the first configuration from the container, and servicing the second request from the container.

[0006] In any embodiment, any or all of the following features may be included in any combination and without limitation: The container may be one of several containers in a multitenant environment that are instantiated to serve requests from client devices. The first configuration may include the size of the heap in memory that can be used by the first service. After flushing the first configuration from the container, the container may include a runtime process having an embedded server and an internal endpoint. The internal endpoint may be invoked by a router in a multitenant environment to serve a second request. The first configuration may include several actions that are linked together to serve the request. A multitenant environment may prevent a container from simultaneously serving requests associated with different tenants. A multitenant environment may allow a container to simultaneously serve requests associated with a single tenant. The method / operation may also include the steps of receiving a third request for a second service provided by a second tenant, and serving the third request from the container without flushing the second configuration from the container. The first service may include a public API that becomes available through the multitenant environment. [Brief explanation of the drawing]

[0007] A further understanding of the nature and merits of the present invention can be achieved by referring to the remainder of the specification and the drawings. The same reference numerals are used across several drawings to refer to similar components. In some cases, a sublabel is associated with a reference numeral to indicate one of several similar components. Where a reference numeral is referenced without specificity to an existing sublabel, it is intended to refer to all such multiple similar components. [Figure 1]This is a simplified block diagram of a system for handling requests in a multi-tenant environment, according to several embodiments. [Figure 2] This is a diagram of a container instance 108 prepared to serve requests to a tenant, according to several embodiments. [Figure 3] This block diagram shows how files from a datastore can be populated into an empty container instance according to several embodiments. [Figure 4] This diagram shows how a submitted container can serve a request according to several embodiments. [Figure 5] This is a simplified block diagram of a container being returned to a pool of available containers, according to several embodiments. [Figure 6A] This is a state diagram of the container lifecycle in a multi-tenant environment, according to several embodiments. [Figure 6B] This is an alternative state diagram of the container lifecycle in a multi-tenant environment, according to several embodiments. [Figure 7] This is a flowchart illustrating how to use containers to isolate tenants in a multi-tenant environment to serve requests. [Figure 8] This is a flowchart illustrating a method for managing runtime state and service configuration for containers in a multi-tenant environment, according to several embodiments. [Figure 9] This is a flowchart illustrating a method for efficiently allocating a pool of containers to serve requests in a multi-tenant environment, according to several embodiments. [Figure 10] This is a simplified block diagram of a distributed system for realizing some of the embodiments. [Figure 11] This is a simplified block diagram of the components of a system environment that can provide services offered by the components of an embodiment system as cloud services. [Figure 12]This figure shows an exemplary computer system in which various embodiments can be realized. [Modes for carrying out the invention]

[0008] Detailed explanation This describes an embodiment for managing an API gateway. A cloud system can host APIs for multiple different tenants and receive requests from many different client devices. When a request is received, the associated tenant can be identified, and a router can determine whether a container instance is available to service the request. A container instance may be an empty container instance containing an internal endpoint, a web server, and a runtime environment. An empty container instance cannot be associated with a particular tenant. To associate a container instance with a tenant, a data store, such as a key-value data store, can look up a configuration file that transforms an agnostic container instance into a container instance that is associated with a particular tenant and contains configuration code to perform the required API functions. Once the API in the container has finished serving the request (or multiple requests for a single tenant), runtime state information can be returned to the data store and stored, and the contents of the container instance can be flushed. This ensures isolation of tenant data in a multi-tenant environment while still allowing for container reuse and efficient management of pooled resources. The datastore can maintain a service registry, enabling the system's router to allocate new container instances when needed and deallocate them when they are not in use. The datastore can also store runtime state information, configurations, and applications for tenant APIs that can be distributed to any container on an on-demand basis.

[0009] Figure 1 is a simplified block diagram of a system for handling requests in a multitenant environment, according to several embodiments. Throughout this disclosure, an example of an API gateway may be used. However, the embodiments described herein are not limited thereto. Instead, the functionality described for handling requests to an API gateway can be implemented in any system having an interface for receiving requests.

[0010] Multiple client devices 102 (e.g., smartphones, laptops, tablet computers, workstations, servers, etc.) can send requests to a public interface 104, which may include a Load Balancer as a Service (LBaaS) interface. These requests may be associated with a specific tenant in a multi-tenant environment and may refer to a specific service provided by that tenant, such as an API function. For example, a request received from one of the client devices 102 may include information such as "Tenant:ACME, API:ProcessInvoice". The public interface 104 receives that request. It can be sent to one or more routers 106.

[0011] Router 106 can communicate periodically and in real time with datastore 112 to determine one or more containers that can be assigned to a tenant and handle a particular request. In the example in Figure 1, a request to tenant 2 would have three different container instances that router 106 can select. Within a set of container instances assigned to tenant 2, router 106 can select one or more container instances loaded with configurations and / or applications to service the request. For example, router 106 could select "Te" loaded with configurations to handle "API:ProcessInvoice". Select one of the three container instances assigned to "nant:ACME". This is possible. If this configuration is not yet loaded in any instance, router 106 can select an existing empty container or instantiate a new empty container. Router 106 can then handle a specific request for "Tenant:ACME". To create a containerized service, datastore 112 can be used to transfer configuration specific to "API:ProcessInvoice" to an existing / new empty container.

[0012] Once a containerized service has handled a request and (if necessary) returned a response to the client device, the containerized service can be reused to serve additional requests for that particular tenant / service combination, or the container, with its contents flushed, can be made available to other tenants. If a container is flushed and reassigned to a different tenant, runtime information that needs to be stored in the container can be sent back to datastore 112. This ensures that the next time that particular application or API is instantiated in an empty container, the runtime information can be transferred and used by the new container to resume from where it left off.

[0013] The registration service 114 can be used to register a new tenant in a multi-tenant environment. The new tenant can use the interface of the registration service 114 to generate a new tenant entry in the data store 112 and define parameters for using that multi-tenant environment. These parameters may include the maximum, minimum, average, and expected number of container instances available to that particular tenant. In addition, the new tenant can select the number and type of routers to service requests, different configuration services that may be available, different configurations of runtime container pools, and other aspects of the registration process.

[0014] The configuration service 110 may provide a web-based user interface that enables a tenant to define a configuration and select applications that can be made available through a public interface. For example, some embodiments may provide a series of web forms and dropdown boxes that enable a tenant to select from multiple different microservices, applications, functions, etc. to build an API. Some embodiments may also enable a tenant to select executable applications, such as an application for limiting the number of times an API can be called within a predetermined time limit. After the configuration and applications are selected / defined through the configuration service 110, they can be stored in the data center 112 for placement into empty containers in the system.

[0015] Note that a given gateway can respond to two or more tenants simultaneously. In some embodiments, there may be multiple gateways that are completely separated and run on different availability domains or data centers. For example, there may be a gateway for Chicago and a gateway for Phoenix. However both of these gateways may still serve the same tenant. An end user may still deploy an API to one or any combination of the multiple gateways using a unique gateway ID. In some embodiments, each gateway may have its own unique key-value data store for storing configurations, applications, etc.

[0016] The following figures illustrate an example of how requests can be processed by the system. These examples are not intended to be limiting. Figure 2 shows a container instance Figure S108 is shown. At this stage, router 106 has already received a request to be served. First, router 106 can work in conjunction with datastore 112 to determine whether available container instances are operational and available to serve the request. Datastore 112 may include a service registry 214 that catalogs each of the available instances in the system. The service registry 214 can also track which APIs and tenants are assigned to each instantiated container. When a new container is instantiated or an existing container is removed, the service registry 214 may be updated in real time by datastore 112. Router 106 may maintain a local copy of at least a portion of the service registry 214. Datastore 112 can intermittently update router 106 with a list of changes to the service registry 214. In some embodiments, datastore 112 can update router 106 with a list of available container instances that can receive the request. In some embodiments, the total list of instances in the system may be very large, so only changes or a portion of the service registry 214 are sent to router 106.

[0017] As shown in Figure 1, some embodiments may include multiple routers 106. One of the routers 106-2 may be designated as the master router. The master router 106-2 may be designated as the only router 106 permitted to instantiate new container instances when necessary. Thus, if router 106 in Figure 2 is designated as the master router, it is free to instantiate new container instances as needed to serve requests for a particular tenant. The master router 106-2 can analyze the configuration provided by the tenant during registration to ensure that the number of container instances allocated to the tenant in the pool is within the minimum / maximum number for that particular tenant. If one of the routers 106 is handling a request that requires a new container to be instantiated, router 106-1 can forward the request to the master router 106-2, which can then determine whether the new container should be instantiated. By allowing only the master router 106-2 to instantiate new containers, this helps ensure that the size of the container pool is managed efficiently. For example, if six different routers all receive simultaneous requests for a particular service for a particular tenant, each of these routers could independently decide whether to instantiate a new container if one instantiated container is sufficient. Sending each of these requests to the master router 106-2 ensures that only the minimum number of new containers are instantiated.

[0018] Conversely, if router 106 and / or datastore 112 determine that multiple unused container instances are allocated to a single tenant, or that multiple unallocated containers are available in the container pool, it can determine that one or more of these unused containers can be removed from the system, thereby freeing up memory and processing power for existing containers to operate. This provides a dynamic pool of container instances that can serve requests. This pool can dynamically grow / shrink based on how much request traffic is received at any given time. Flushing container content, as described below, allows containers to be reused across different tenants while still ensuring that tenants are isolated from one another.

[0019] One technical advantage achieved by these embodiments is the efficient management of container pool size. A runtime pool may be a collection of runtime instances that share all common characteristics. A runtime instance in a container pool is a given runtime instance Instances may be numbered sequentially, starting at index 0, so that they are uniquely identified by a pair of values: {pool name, instance index}. Pool management may depend on the specific container environment used by each embodiment. For example, a Java® class may be used to implement pool functionality using the Docker daemon REST API to create / start / stop runtime containers as needed. This class may allow the environment to configure the pool name, the number of ports, the number of debug ports, the container state transition timeout value (described in detail below), etc. Some embodiments may also allow this class to specify a “minimum size” value representing the number of tenant-unbound running container instances that would be available to serve requests. In addition, a “maximum size” value may specify the maximum number of runtime container instances in the container pool.

[0020] Once a service is assigned to a container in the gateway, the service may perform periodic "heartbeats" as an indicator to other services that it is alive and functioning properly. For example, when a service loads a container If so, it may perform a heartbeat to inform the router that it is available to serve the request. Performing a heartbeat may involve updating the corresponding entry in the service registry of datastore 112. These heartbeat transmissions may be used to determine the lifecycle of the container at the gateway. For example, some embodiments may use a time-to-live (TTL) interval after which the service may be considered inactive. Alternatively, some embodiments may specify a number of heartbeats that are allowed to be skipped / overlooked before the service is considered inactive. Heartbeats may occur periodically, such as every 10 seconds, every 20 seconds, every 30 seconds, every 60 seconds, etc.

[0021] In the example in Figure 2, router 106 can identify empty container 108-3, which is not currently assigned to a specific tenant, or is assigned to the requesting tenant but has not been configured to execute the requesting specific API. Containers 108-1 and 108-2 are configured and assigned to specific tenants. Since they operate in separate containers from container 108-3, the data and operations of these three containers can be strictly isolated from each other.

[0022] Figure 3 shows a block diagram illustrating how files from datastore 112 may be populated into an empty container instance according to several embodiments. Continuing the example in Figure 2, an empty container 108-3 may be assigned to handle request 304 for a particular tenant. First, container 108-3 may be assigned to that tenant so that it cannot serve requests from other tenants through container 108-3. Next, tenant and API information from request 304 may be used to look up configuration, application, and runtime information in datastore 112. In some embodiments, datastore 112 may be a key-value datastore. In some embodiments, datastore 112 may also be distributed across different systems in a multi-tenant environment or across different platforms. For example, the tenant and / or API from request 304 may be used as a key to look up values ​​in datastore 112 that return configuration 202, application 204, and / or arbitrary runtime state information.

[0023] In some embodiments, the key-value datastore 112 may be used to persist tenant-specific configurations to disk. In addition, the key-value datastore is used for all running A central service registry 214 can be provided so that each microservice "registers" itself and other services can locate and start them. For example, a distributed key-value datastore such as etcd® may be used to store state information such as configurations ingested from the management service (e.g., policies, APIs, applications, planning metadata, etc.); runtime container state in the service registry 214; container pool configuration (e.g., minimum, maximum, strategy, timeout interval, etc.); tenant registration status (e.g., tenants, tenant-pool associations, etc.); rate-limiting configurations, real-time state, etc.

[0024] Configuration 202 may include a pipeline of actions 206 defined in Configuration Service 110 in Figure 1. These actions 206 may be linked together to form other services for processing an API or data. Each of the actions 206 may include receiving a request, parsing the payload in the request, processing the data in the payload, modifying the data in the payload, calling another service to obtain information, writing information to a file or database, and / or generating a response. In some embodiments, Configuration 202 can handle multiple requests simultaneously. Thus, a single container in which Configuration 202 is deployed can handle multiple requests for that particular API and its particular tenant. Generally, configurations are stateless (e.g., RESTful) and highly efficient in processing information and generating responses quickly.

[0025] In addition to returning the configuration 202, the datastore 112 can return one or more applications 204 that can run in a container along with the API configuration 202. For example, application 204 may be a bandwidth-limited application that limits the number of times a particular API can be called within a predetermined time interval (for example, only 100 requests per hour may be served). In contrast to the configuration 202, application 204 may require runtime state information to be saved between application executions. In Figure 3, application 204 may not have been executed in the past, and therefore there may be no state runtime information saved in the datastore 112 for application 204. Container 108-3 here may represent a microservice assigned to a single tenant to handle a particular API request. Once container 108-3 is joined to a tenant, the instance can load all new configuration information, such as API definitions, applications, plans, and subscriptions, from the datastore 112 on demand.

[0026] An empty container may be a software container such as a Docker® container, and a multitenant environment may include an orchestrated container platform such as Kubernetes®. Instantiating a new empty container may include generating an empty container from a container image and populating it with a minimum number of software processes that would be common to any configuration used in the system. For example, some embodiments may specify an empty container as a Docker® container including a runtime environment 212 such as a Java Runtime Environment (JRE), a web server 210 such as an HTTP server, and an internal endpoint 208. The internal endpoint 208 may be exposed to a router 106 and can be used by the router 106 to send requests to container 108-3. In some embodiments, a container environment other than a Docker® container may be used. For example, some embodiments may use UNIX® processes to start / stop the runtime container.

[0027] When an empty container is loaded, datastore 112 will configure 202 and the application The configuration 204 can be transferred to container 108-3. If runtime state data 302 is available in datastore 112, it will also be transferred to container 108-3. In this case, runtime state data 302 is generated when the application is executed and stored in container 108-3. For example, application 204 may record the number of requests received within a given time interval. This information is stored in runtime state 302 and can be transferred back to datastore 112 when this container 108-3 is flushed. In general, transferring the configuration 202, application 204, and / or runtime state 302 to an empty but instantiated container 108-3 is a relatively lightweight process that can be done very quickly and efficiently to handle requests with little delay.

[0028] Figure 4 illustrates how a spawned container 108-3 can service requests according to several embodiments. Container 108-3 can service any request for this tenant for the API defined by configuration 202. In some cases, this may involve processing only a single request that configuration 202 has forwarded to container 108-3. In other cases, this may involve processing multiple similar requests for the same tenant that have been sent to router 106. After all requests have been processed and responses (if any) have been sent back to the requesting client device, container 108-3 can become idle or passive. While container 108-3 is still assigned to or bound to a particular tenant, it is not currently being used to process any requests. After a predetermined time interval, container 108-3 may be deallocated from that particular tenant and returned to a pool of available containers 108-3 awaiting assignment to a new tenant with a new configuration.

[0029] Any runtime state information 302 generated or updated by application 204 running on container 108-3 may be stored in datastore 112 before container 108-3 is flushed and reassigned to a different tenant. Then, if configuration 202 and / or application 204 are reassigned to a new container to serve future requests, the runtime state information 302 may be sent to the different container.

[0030] Figure 5 shows a simplified block diagram of container 108-3 being returned to the pool of available containers according to several embodiments. After the runtime state information 302 is transferred to and returned to the data store 112, the configuration 202, application 204, and / or runtime state 302 may be flushed from container 108-3. The empty container 108-3 can now be reassigned to a different tenant by router 106 to serve a different API call. Although Figure 5 illustrates only container 108-3 as empty, an actual deployment may typically include multiple empty containers. Router 106 can use different strategies to assign requests to one of the multiple available empty containers, such as a brute-force strategy. As described above, if container 108-3 is not assigned to a new tenant / API within a predetermined time interval, container 108-3 can be removed from the multi-tenant environment to conserve memory and / or computing resources.

[0031] As described above, datastore 112 facilitates these operations by distributing configuration, applications, and runtime state across various containers operating in a multi-tenant environment. Datastore 112 may also receive configuration information from tenants upon registration and even at runtime. Datastore 112 also maintains a service registry 214 that monitors the state of the container pool at all times. Datastore 112 uses this service registry 214 to communicate with router 106 to determine when the pool of available containers should grow and / or shrink.

[0032] Figure 6A shows a state diagram of the container lifecycle in a multi-tenant environment according to several embodiments. In the initial state 602, the container does not exist. In state 604, the container has already been instantiated with the set of processes described above (e.g., web server, endpoint, etc.), but the container is empty, not bound to or assigned to a particular tenant. To service requests, the container can enter state 606, where it is bound to or assigned to a particular tenant, and its configuration, applications, and / or runtime state information are populated. State 606 is called active because the container can actively service requests received from router 106. In state 608, the container is still passive, meaning it can still be bound to or assigned to a particular tenant, but it is not actively processing requests using the configuration stored within it. After remaining idle for a predetermined time interval, the container can be assigned or unassigned in state 610. If the container is no longer bound to a particular tenant, its internal configuration, applications, and / or runtime state can be flushed. In some embodiments, an unbound container does not need to flush its internal contents until it is reassigned to a new tenant. If the container is not assigned to a new tenant, the container may be removed in state 612.

[0033] Figure 6B shows another state diagram from Figure 6A illustrating the container lifecycle in a multi-tenant environment according to several embodiments. In the initial state 602, the container does not yet exist or has already been removed from the container environment. Once a container instance is created, it may be unbound in state 604. If a container is idle for a predetermined time interval called the "unbound timeout," the container may be removed from the environment and return to state 602. Alternatively, in state 606, the container may be assigned to a tenant and loaded with configuration, applications, state information, etc. From bound and active state 606, the container may be removed from the environment if the router reduces the size of the container pool and thus transitions back to state 602. The container may also be released from the tenant in state 604, flush its tenant-specific content, and return to the unbound pool of containers. Furthermore, a bound and active container in state 606 may become passive in state 608 if it remains idle without servicing client requests for a predetermined time interval called the "idle tenant timeout." From coupled passive state 608, the "passive timeout" interval may expire, causing the container to transition from coupled passive container state 608 to the uncoupled pool of containers in state 604. As long as the container exists, this container lifecycle can transition between states.

[0034] Figure 7 shows a flowchart of a method for isolating tenants using containers to serve requests in a multi-tenant environment. The method may include the step (702) of receiving a first request for a first service provided by a first tenant. The method may also include the step (704) of selecting an empty container in the multi-tenant environment. The method may further include the step (706) of loading a first configuration that realizes the first service into the container. The method may further include the step (708) of servicing the first request from the container. The method may also include the step (710) of receiving a second request for a second service provided by a second tenant. The method may further include the step (712) of flushing the first configuration from the container. The method may further include the step (714) of servicing the second request from the container. Some embodiments may include a system including one or more processors and one or more memories to perform these method steps. Other embodiments may include a system that directs one or more processors to perform these method steps. It may include a non-temporary computer-readable medium for storing the orders.

[0035] In any embodiment, one or more of the following features may be included in any combination and without limitation: The container may be one of several containers in a multitenant environment that are instantiated to serve requests from client devices. After flushing a first configuration from the container, the container may include a runtime process having an embedded server and an internal endpoint. The internal endpoint may be invoked by a router in the multitenant environment to serve a second request. The first configuration may include several actions that are linked together to serve a request. The multitenant environment may prevent a container from simultaneously serving requests associated with different tenants. The multitenant environment may allow a container to simultaneously serve requests associated with a single tenant. The method may also include the steps of receiving a third request for a second service provided by a second tenant, and serving the third request from the container without flushing the second configuration from the container. The first service may include a public API that becomes available through the multitenant environment.

[0036] It should be understood that the specific steps shown in Figure 7 provide a particular method for isolating tenants from each other using containers to serve requests in a multi-tenant environment, according to various embodiments of the present invention. Steps in other sequences may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, the individual steps shown in Figure 7 may include a number of substeps that can be performed in various orders as appropriate for the individual step. In addition, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0037] Figure 8 shows a flowchart of a method for managing runtime state and service configurations for a container in a multitenant environment, according to several embodiments. The method may include a step (802) of receiving an indication that a request has been received for a service provided in the multitenant environment. The method may also include a step (804) of identifying a configuration that provides the service and storing the configuration in a datastore. The method may also include a step (806) of sending the configuration to a container in the multitenant environment to service the request. The method may further include a step (808) of receiving runtime state from the container. The method may also include a step (810) of storing the runtime state in a datastore and flushing the configuration from the container. Some embodiments may include a system including one or more processors and one or more memories to perform these method steps. Other embodiments may include a non-temporary computer-readable medium that stores instructions causing one or more processors to perform these method steps.

[0038] In any embodiment, one or more of the following features may be included in any combination and without limitation: The configuration may be provided by tenants in a multitenant environment prior to runtime. The datastore may include a key-value datastore. The key-value datastore may include a distributed key-value datastore. The identity of a tenant associated with a service may be a key in the key-value datastore, and the configuration and runtime state may be values ​​in the key-value datastore. The datastore may also store a registry of containers available in the multitenant environment. When a new container becomes available in the multitenant environment, the datastore, based on the container registry, will store the registry of containers in the multitenant environment. You may replace one or more routers.

[0039] It should be understood that the specific steps shown in Figure 8 provide a particular method for managing runtime state and service configuration for containers in a multitenant environment, according to various embodiments. Steps in other sequences may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, the individual steps shown in Figure 8 may include multiple substeps that can be performed in various orders as appropriate for the individual step. In addition, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0040] Figure 9 shows a flowchart of a method for efficiently allocating a pool of containers to serve requests in a multi-tenant environment, according to several embodiments. The method may include the step (902) of allocating a plurality of containers to a first tenant in the multi-tenant environment. The method may also include the step (904) of identifying one or more containers in the plurality of containers that are allocated to the first tenant but are not being used by the first tenant. The method may further include the step (906) of flushing the contents of one or more containers. The method may further include the step (908) of reassigning one or more containers to a second tenant in the multi-tenant environment. Some embodiments may include a system including one or more processors and one or more memories to perform these method steps. Other embodiments may include a non-temporary computer-readable medium that stores instructions causing one or more processors to perform these method steps.

[0041] In any embodiment, one or more of the following features may be included in any combination and without limitation: After flushing the contents of one or more containers, one or more containers do not need to be assigned to any tenant for a first time interval before being reassigned to a second tenant. The method may also include the steps of identifying one or more second containers in a plurality of containers that are assigned to a first tenant but are not being used by the first tenant, determining that no other tenants require one or more second containers, and removing one or more second containers from the multitenant environment. The method may further include the step of determining that the first tenant is receiving more requests than can be served by the plurality of containers. The method may further include the steps of instantiating a new plurality of containers and assigning the new plurality of containers to the first tenant. The method may also include the step of assigning containers previously assigned to another tenant to the first tenant. A gateway in a multitenant environment may reassign one or more containers to a second tenant in the multitenant environment.

[0042] It should be understood that the specific steps shown in Figure 9 provide a particular method for efficiently allocating a pool of containers to serve requests in a multi-tenant environment, according to various embodiments. Steps in other sequences may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, the individual steps shown in Figure 9 may include a number of substeps that can be performed in various orders as appropriate for the individual step. In addition, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0043] Each of the methods described herein may be implemented by a computer system. Each step of these methods may be performed automatically by a computer system and / or may involve user inputs / outputs. For example, a user may provide inputs at each step of the method, each of which may respond to a specific output requesting such input, and the output may be generated by the computer system. Each input may be received in response to a corresponding requested output. Furthermore, inputs may be received from the user as a data stream from another computer system, retrieved from memory locations, retrieved over a network, or requested from a web service. Similarly, outputs may be provided to the user as a data stream from another computer system, stored in memory locations, transmitted over a network, or provided to a web service. In short, each step of the methods described herein may be performed by a computer system and may involve any number of inputs, outputs, and / or requests to and from the computer system, with or without user involvement. Those steps without user involvement may be said to be performed automatically by the computer system without human intervention. Therefore, it will be understood in light of this disclosure that each step of each method described herein may be modified to include input and output with a user, or may be performed automatically by a computer system without human intervention, in which case all decisions are made by the processor. Furthermore, some embodiments of each method described herein may be implemented as a set of instructions stored on a tangible, non-temporary storage medium to form a tangible software product.

[0044] Figure 10 shows a simplified diagram of a distributed system 1000 for realizing one of the embodiments. In the illustrated embodiment, the distributed system 1000 includes one or more client computing devices 1002, 1004, 1006, and 1008, which are configured to run and operate client applications such as web browsers and dedicated clients (e.g., Oracle Forms) through one or more networks 1010. A server 1012 may be coupled to the remote client computing devices 1002, 1004, 1006, and 1008 via the network 1010 so as to be able to communicate with them.

[0045] In various embodiments, the server 1012 may be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services may be web-based services or cloud services, or under a Software as a Service (SaaS) model, providing client computing support. These may be provided to users of the computing devices 1002, 1004, 1006, and / or 1008. Users operating the client computing devices 1002, 1004, 1006, and / or 1008 may then use one or more client applications to interact with the server 1012 in order to use the services provided by these components.

[0046] In the configuration shown in the figure, software components 1018, 1020, and 1022 of system 1000 are shown as being implemented on server 1012. In other embodiments, one or more of the components of system 1000 and / or the services provided by these components may also be implemented by one or more of the client computing devices 1002, 1004, 1006, and / or 1008. A user operating a client computing device may then utilize one or more client applications to use the services provided by these components. These components may be implemented as hardware, firmware, software, or a combination thereof. Distributed System It should be understood that various different system configurations, which may differ from 1000, are possible. Therefore, the embodiment shown in the figure is merely an example of a distributed system for realizing the embodiment system and is not intended to be limiting.

[0047] Client computing devices 1002, 1004, 1006, and / or 1008 may be portable handheld devices (e.g., iPhone®, mobile phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass® head-mounted displays), running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android®, BlackBerry® 10, PalmOS, and providing internet, email, short message service (SMS), BlackBerry®, or other communication services. It may also support Rotor. Client computing devices can be general-purpose personal computers, including, for example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. Client computing devices may also run various GNU / Linux operating systems, such as Google Chrome OS. The workstation computer may run any of the various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, the UNIX operating system. Alternatively, or in addition to, the client computing devices 1002, 1004, 1006, and 1008 may be any other electronic devices that can communicate through the network 1010, such as thin client computers, internet-enabled gaming systems (e.g., Microsoft Xbox gaming consoles with or without Kinect® gesture input devices), and / or personal messaging devices.

[0048] Although the exemplary distributed system 1000 is shown having four client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 1012.

[0049] Network 1010 in distributed system 1000 uses TCP / IP (Transmission Control Protocol / Internet Protocol). Network 1010 may be any type of network well known to those skilled in the art, capable of supporting data communication using any of the various commercially available protocols, including, but not limited to, CAL, SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. For example, network 1010 could be a local area network (LAN), such as one based on Ethernet®, Token Ring, etc. Network 1010 could be a wide area network or the Internet. It could also be a virtual private network (VPN), intranet Net, extranet, public switched telephone network (PS) TN), infrared networks, wireless networks (for example, the Institute of Electrical and Electronics (IEEE) 802.11 protocol suite) Bluetooth® and / or any other wireless protocol This may include virtual networks, which may include (networks operating under the above), and / or any combination of these and / or other networks, without limitation.

[0050] Server 1012 may consist of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, UNIX® servers, midrange servers, mainframe computers, rack-mount servers, etc.), server farms, server clusters, or any other suitable configuration and / or combination. In various embodiments, Server 1012 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, Server 1012 may correspond to a server for performing the processing described above according to one embodiment of this disclosure.

[0051] Server 1012 may run an operating system including any of the above, and any commercially available server operating system. Server 1012 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP (hypertext transport protocol) servers, FTP (file transfer protocol) servers, CGI (common gateway interface) servers, JAVA® servers, database servers, etc. Illustrative database servers include Oracle, Microsoft, Sybase, IBM (International Business Machines) This includes, without limitation, anything commercially available from sources such as (Likely a list of brands or other retailers).

[0052] In some implementations, server 1012 may include one or more applications for analyzing and integrating data feeds and / or event updates received from users of client computing devices 1002, 1004, 1006, and 1008. For example, data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams, which may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc. Server 1012 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 1002, 1004, 1006, and 1008.

[0053] The distributed system 1000 may also include one or more databases 1014 and 1016. Databases 1014 and 1016 may reside in various locations. For example, one or more of databases 1014 and 1016 may reside on a non-temporary storage medium local to (and / or located within) server 1012. Alternatively, databases 1014 and 1016 may be remote from server 1012 and communicate with server 1012 via a network-based connection or a dedicated connection. In one embodiment, databases 1014 and 1016 may reside on a storage area network (SAN). Similarly, any necessary files for performing functions attributable to server 1012 may be stored locally on server 1012 and / or remotely, as appropriate. In one embodiment, databases 1014 and 1016 are configured to store, update, and retrieve data in response to commands in SQL format. It may include a combined relational database, such as a database provided by Oracle.

[0054] Figure 11 is a simplified block diagram of one or more components of a system environment 1100 that can provide services as cloud services, provided by one or more components of the embodiment system according to one embodiment of the present disclosure. In the illustrated embodiment, the system environment 1100 includes one or more client computing devices 1104, 1106, and 1108 that can be used by a user to interact with a cloud infrastructure system 1102 that provides cloud services. These client computing devices may be configured to run client applications, such as a web browser, a dedicated client application (e.g., Oracle Forms), or any other application, which can be used by a user of the client computing device to interact with the cloud infrastructure system 1102 in order to use the services provided by the cloud infrastructure system 1102.

[0055] It should be understood that the cloud infrastructure system 1102 shown in the figure may have components other than those shown. Furthermore, the embodiment shown in the figure is merely one example of a cloud infrastructure system that may incorporate one embodiment of this invention. In some other embodiments, the cloud infrastructure system 1102 may have more or fewer components than those shown, may be a combination of two or more components, or may have components in different configurations or arrangements.

[0056] Client computing devices 1104, 1106, and 1108 may be similar to those described above for 1002, 1004, 1006, and 1008.

[0057] Although the exemplary system environment 1100 is shown having three client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the cloud infrastructure system 1102.

[0058] Network 1110 may facilitate the communication and exchange of data between clients 1104, 1106, and 1108 and the cloud infrastructure system 1102. Each network may be any type of network well known to those skilled in the art, capable of supporting data communication using any of the various commercially available protocols, including those described above for network 1010.

[0059] The cloud infrastructure system 1102 may include one or more computers and / or servers that may include the above-described components for server 1012.

[0060] In one embodiment, the services provided by the cloud infrastructure system may include a number of services made available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, and managed technical support services. The services provided by the cloud infrastructure system may be dynamically scaled to meet the needs of its users. A specific instantiation of the services provided by the cloud infrastructure system is described here as follows: This is called a "service instance." Generally, any service made available to users via a communication network such as the internet from a cloud service provider's system is called a "cloud service." Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own on-premises servers and systems. For example, the cloud service provider's system may host applications, and users may order and use applications on demand via a communication network such as the internet.

[0061] In some examples, services in a computer network cloud infrastructure may include secure computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by the cloud vendor or otherwise publicly known in the art. For example, a service may include password-protected access to remote storage on the cloud over the internet. Another example is a service that includes a web service-based hosted relational database and scripting language middleware engine for private use by networked developers. Yet another example is access to an email software application hosted on the cloud vendor's website.

[0062] In one embodiment, the cloud infrastructure system 1102 may include a set of applications, middleware, and database service offerings delivered to customers in a self-service, subscription-based, resilient, scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud provided by the Assignee. It is a public cloud.

[0063] In various embodiments, the cloud infrastructure system 1102 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by the cloud infrastructure system 1102. The cloud infrastructure system 1102 may provide cloud services through different deployment models. For example, the services may be provided under a public cloud model, where the cloud infrastructure system 1102 is owned by an organization that sells cloud services (e.g., owned by Oracle), and the services are made available to the general public or different industrial enterprises. As another example, the services may be provided under a private cloud model, where the cloud infrastructure system 1102 operates for only one organization and can provide services for one or more entities within that organization. The cloud services may also be provided under a community cloud model, where the cloud infrastructure system 1102 and the services provided by the cloud infrastructure system 1102 are shared by several organizations in the relevant community. The cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0064] In some embodiments, the services provided by the cloud infrastructure system 1102 fall under the categories of Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), or One or more services offered under other categories of services, including hybrid services. It may include bis. The customer may order one or more services provided by the cloud infrastructure system 1102 via a subscription order. The cloud infrastructure system 1102 then processes the services in the customer's subscription order.

[0065] In some embodiments, the services provided by the cloud infrastructure system 1102 may include, without limitation, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall under the SaaS category. For example, the SaaS platform may provide the ability to build and deliver a set of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing the SaaS services. By using the services provided by the SaaS platform, customers can utilize applications running on the cloud infrastructure system. Customers can obtain application services without having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, without limitation, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0066] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall under the PaaS category. Examples of platform services may include, without limitation, services that enable organizations (such as Oracle) to integrate existing applications on a shared common architecture, and the ability to build new applications that leverage the shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure for providing the PaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, without limitation, Oracle Java® Cloud Service (JCS), Oracle Database Cloud Service (DBCS), and others.

[0067] By utilizing the services provided by the PaaS platform, customers can adopt programming languages ​​and tools supported by the cloud infrastructure system, as well as control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model that enables an organization to pool database resources and provide customers with a database-as-a-service in the form of a database cloud. The middleware cloud service may provide a platform for customers to develop and deploy various business applications, and the Java cloud service may provide a platform for customers to deploy Java applications on the cloud infrastructure system.

[0068] In a cloud infrastructure system, various different infrastructure services may be provided by an IaaS platform. These infrastructure services facilitate the management and control of basic computing resources, such as storage, networking, and other fundamental computing resources, for customers using services provided by SaaS and PaaS platforms.

[0069] In one embodiment, the cloud infrastructure system 1102 may also include infrastructure resources 1130 for providing resources used to provide various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1130 may include a pre-integrated and optimized combination of hardware such as servers, storage, and networking resources for running services provided by the PaaS platform and SaaS platform.

[0070] In some embodiments, resources in the cloud infrastructure system 1102 may be shared by multiple users and dynamically redistributed according to their needs. In addition, resources may be allocated to users in different time zones. For example, the cloud infrastructure system 1130 may allow a first set of users in a first time zone to utilize the resources of the cloud infrastructure system for a certain number of hours, and then allow the same resources to be reallocated to another set of users located in a different time zone, thereby maximizing resource utilization.

[0071] In one embodiment, a number of internal shared services 1132 may be provided, shared by different components or modules of the cloud infrastructure system 1102 and by the services provided by the cloud infrastructure system 1102. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, and the like.

[0072] In one embodiment, the cloud infrastructure system 1102 may provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, the cloud management functionality may include the ability to provision, manage, and track customer subscriptions received by the cloud infrastructure system 1102.

[0073] In one embodiment, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 1120, an order orchestration module 1122, an order provisioning module 1124, an order management and monitoring module 1126, and an identity management module 1128. These modules may include, or be provided using, one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable configuration and / or combination.

[0074] In example operation 1134, client computing devices 1104, 11 Customers using client devices such as 06 or 1108 may interact with the cloud infrastructure system 1102 by requesting one or more services provided by the cloud infrastructure system 1102 and by placing subscription orders for one or more services provided by the cloud infrastructure system 1102. In one embodiment, the customer may access a cloud user interface (UI), cloud UI 1112, cloud UI 1114 and / or cloud UI 1116 and place subscription orders through these UIs. The order information received by the cloud infrastructure system 1102 in response to the customer's order may include information identifying the customer and one or more services provided by the cloud infrastructure system 1102 that the customer intends to subscribe to.

[0075] After an order is placed by a customer, the order information is received via the cloud UI 1112, 1114, and / or 1116.

[0076] In operation 1136, the order is stored in the order database 1118. The order database 1118 is operated by the cloud infrastructure system 1118 and may be one of several databases operating together with other system elements.

[0077] In operation 1138, order information may be sent to the order management module 1120. In some cases, the order management module 1120 may be configured to perform billing and charging functions related to the order, such as verifying the order and reserving the order after verification.

[0078] In operation 1140, information regarding the order may be communicated to the order orchestration module 1122. The order orchestration module 1122 may use the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some cases, the order orchestration module 1122 may orchestrate the provisioning of resources to support the subscription service using the services of the order provisioning module 1124.

[0079] In one embodiment, the order orchestration module 1122 enables the management of the business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. In operation 1142, upon receiving a new subscription order, the order orchestration module 1122 sends a request to the order provisioning module 1124 asking for resources to be allocated and configured as needed to fulfill the subscription order. The order provisioning module 1124 enables the allocation of resources for the service ordered by the customer. The order provisioning module 1124 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 1100 and the physical implementation layer used to provision resources to provide the requested service. The order orchestration module 1122 may therefore be decoupled from implementation details such as whether services and resources are actually provisioned on the fly or pre-provisioned and allocated / assigned only on demand.

[0080] In operation 1144, once the services and resources are provisioned, the notification of the provided services is sent to the cloud infrastructure system 1102 in order provisioning. The Jonning module 1124 may transmit to the customer on client devices 1104, 1106, and / or 1108.

[0081] In operation 1146, customer subscription orders may be managed and tracked by the order management and monitoring module 1126. In some cases, the order management and monitoring module 1126 may be configured to collect utilization statistics about the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0082] In some embodiments, the cloud infrastructure system 1100 may include an identity management module 1128. The identity management module 1128 may be configured to provide identity services in the cloud infrastructure system 1100, such as access management and authentication services. In some embodiments, the identity management module 1128 may control information about customers who wish to use the services provided by the cloud infrastructure system 1102. Such information may include information that authenticates the identity of such customers and information that describes what actions those customers are authorized to take with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1128 may also include managing descriptive information about each customer and descriptive information about who can access and modify that descriptive information and how.

[0083] Figure 12 shows an exemplary computer system 1200 in which various embodiments of the present invention can be realized. System 1200 may be used to realize any of the computer systems described above. As shown in the figure, computer system 1200 includes a processing unit 1204 that communicates with many peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an I / O subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. The storage subsystem 1218 includes a tangible computer-readable storage medium 1222 and a system memory 1210.

[0084] The bus subsystem 1202 provides a mechanism for various components and subsystems of the computer system 1200 to communicate with each other as intended. Although the bus subsystem 1202 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1202 may be one of several types of bus structures, including a memory bus or memory controller, peripheral bus, and local bus, using one of various bus architectures. For example, such architectures include the Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, and Video Electronics Standards Association (VESA) bus. It may include a local bus and a Peripheral Component Interconnect (PCI) bus, which are manufactured according to the IEEE P1386.1 standard. It can be implemented as a mezzanine bus.

[0085] The processing unit 1204, which can be implemented as one or more integrated circuits (for example, conventional microprocessors or microcontrollers), controls the operation of the computer system 1200. The processing unit 1204 may include one or more processors. These processors The processor may include a single-core or multi-core processor. In one embodiment, the processing unit 1204 may be implemented as one or more independent processing units 1232 and / or 1234, each having a single or multi-core processor included in its respective processing unit. In other embodiments, the processing unit 1204 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0086] In various embodiments, the processing unit 1204 can execute various programs in response to program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in the processor 1204 and / or the memory subsystem 1218. Through suitable programming, the processor 1204 can provide the various functionalities described above. The computer system 1200 may also include a processing acceleration unit 1206, which may include a digital signal processor (DSP), a dedicated processor, and the like.

[0087] The I / O subsystem 1208 may include a user interface input device and a user interface output device. The user interface input device may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen embedded in a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. The user interface input device may also include a motion sensing and / or gesture recognition device such as a Microsoft Kinect® motion sensor, which enables the user to control and interact with an input device such as a Microsoft Xbox® 360 game controller through a natural user interface using gestures and verbal commands. The user interface input device may also include an eye gesture recognition device such as a Google Glass® blink detector, which detects eye activity from the user (e.g., blinking while taking a photo and / or selecting a menu) and translates the eye gestures as input to an input device (e.g., Google Glass®). In addition, the user interface input device may include a voice recognition sensing device that allows the user to interact with a voice recognition system (e.g., Siri® Navigator) through voice commands.

[0088] User interface input devices may also include, without limitation, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. In addition, user interface input devices may also include medical imaging input devices such as computed tomography scanners, magnetic resonance imaging scanners, positron emission tomography scanners, and medical ultrasound scanners. User interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0089] The user interface output device may include non-visual displays such as a display subsystem, indicator lights, or audio output devices. The display subsystem may be a flat panel device such as one using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, or a projection device. This may include devices, touchscreens, etc. In general, the use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from the computer system 1200 to a user or another computer. For example, user interface output devices may include, without limitation, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0090] The computer system 1200 may include a storage subsystem 1218 containing software elements, which are shown to be currently located in the system memory 1210. The system memory 1210 may store program instructions that can be loaded and executed on the processing unit 1204, as well as data generated during the execution of these programs.

[0091] Depending on the configuration and type of the computer system 1200, the system memory 1210 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically includes data and / or program modules that are immediately accessible to the processing unit 1204 and / or currently being operated and executed by the processing unit 1204. In some implementations, the system memory 1210 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system may be included, which includes basic routines that help transfer information between elements within the computer system 1200, such as during startup. The system (BIOS) may typically be stored in ROM. (Example, not for limitation) System memory 1210 also indicates application programs 1212, which may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), program data 1214, and operating systems 1216. For example, operating systems 1216 may include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0092] The storage subsystem 1218 may also provide a tangible computer-readable storage medium for storing data structures and basic programming that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by the processor, provides the functionality described above may be stored in the storage subsystem 1218. These software modules or instructions may be executed by the processing unit 1204. The storage subsystem 1218 may also provide a repository for storing data used in accordance with the present invention.

[0093] The storage subsystem 1200 may also include a computer-readable storage medium reader 1220 which can be further connected to the computer-readable storage medium 1222. Together with the system memory 1210, and optionally in combination with the system memory 1210, the computer-readable storage medium 1222 can temporarily and / or otherwise store computer-readable information in addition to remote, local, fixed, and / or removable storage devices. This may comprehensively represent a storage medium for permanently containing, storing, transmitting, and retrieving data.

[0094] The computer-readable storage medium 1222 containing code or a portion of code may also include any suitable medium known or used in the art, including storage and communication media such as volatile and non-volatile, removable and non-removable media, implemented in any method or technique for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices, or other magnetic storage devices, or other tangible computer-readable media. This may also include intangible computer-readable media such as data signals, data transmissions, or any other media that can be used to transmit desired information and are accessible by the computing system 1200.

[0095] For example, the computer-readable storage medium 1222 may include a hard disk drive that reads to and writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads to and writes to a removable non-volatile magnetic medium, and an optical disk drive that reads to and writes to a removable non-volatile optical disk such as a CD-ROM, DVD, Blu-ray® disc, or other optical medium. The computer-readable storage medium 1222 may also include, but is not limited to, Zip® drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD discs, digital videotapes, etc. The computer-readable storage medium 1222 may also include flash memory-based solid-state drives (SSDs), enterprise flash drives, solid-state This may include SSDs based on non-volatile memory such as ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, and magnetoresistive RAM (MRAM) SSDs, as well as hybrid SSDs that use a combination of DRAM-based SSDs and flash memory-based SSDs. Disk drives and their associated computer-readable media may provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data for the computer system 1200.

[0096] The communication subsystem 1224 provides interfaces to other computer systems and networks. The communication subsystem 1224 functions as an interface for receiving data from systems other than computer system 1200 and transmitting data to other systems. For example, the communication subsystem 1224 may allow computer system 1200 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1224 provides radio frequency (RF) transceivers for accessing radio voice and / or data networks (using, for example, cellular technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution), WiFi (IEEE 802.11 family standards), or other mobile communication technologies, or any combination thereof, or advanced data network technologies). Components, Global Positioning System (GPS) receiver component It may include a network and / or other components. In some embodiments, the communication subsystem 1224 can provide a wired network connection (e.g., Ethernet) in addition to, or instead of, a wireless interface.

[0097] In some embodiments, the communication subsystem 1224 may also receive input communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc., for one or more users of the computer system 1200.

[0098] For example, the communication subsystem 1224 handles Twitter® feeds, Facebook® updates, and Rich Site Summary (RSS) feeds. It may be configured to receive data feeds 1226 in real time from users of social networks and / or other communication services, such as web feeds like .edo and / or real-time updates from one or more third-party sources.

[0099] In addition, the communication subsystem 1224 may also be configured to receive data in the form of a continuous data stream that may have no definite end and can be inherently continuous or infinite, which may include an event stream 1228 and / or event updates 1230 of real-time events. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0100] The communication subsystem 1224 may also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 1200.

[0101] The computer system 1200 may be one of a variety of types, including handheld mobile devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0102] Due to the constantly changing nature of computers and networks, the description of the computer system 1200 shown in the figure is intended to be merely a specific example. Many other configurations are possible, having more or fewer components than the system shown in the figure. For example, customized hardware may also be used, and / or certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosures and teachings provided herein, those skilled in the art will understand other ways and / or methods for implementing various embodiments.

[0103] In the preceding description, many specific details have been stated for illustrative purposes to provide a complete understanding of the various embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention can be practiced without some of these specific details. In other examples, well-known structures and devices are shown in the form of block diagrams.

[0104] The foregoing description provides only exemplary embodiments and is not intended to limit the scope, availability, or configuration of this disclosure. Rather, the foregoing description of exemplary embodiments will provide a practicable explanation for realizing the exemplary embodiments for those skilled in the art. Without departing from the spirit and scope of this invention as set forth in the appended claims, elements It should be understood that various changes may be made to the functionality and arrangement of the system.

[0105] The above description provides certain details in order to give a complete understanding of the embodiments. However, it will be understood by those skilled in the art that the embodiments can be practiced even without these specific details. For example, circuits, systems, networks, processes, and other components may have been shown as components in the form of block diagrams so as not to obscure the embodiments by describing them in excessive detail. In other examples, well-known circuits, processes, algorithms, structures, and techniques may have been shown without unnecessary details so as not to obscure the embodiments.

[0106] Furthermore, it should be noted that individual embodiments may have been described as processes shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may have described operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when its operation is complete, but it may have additional steps not shown in the diagram. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, its termination may correspond to that function returning to the calling function or main function.

[0107] The term “computer-readable medium” includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other media that can store, contain, or carry instructions and / or data. A code segment or machine-executable instruction may represent a procedure, function, subprogram, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, sent, or transmitted via any preferred means, including memory sharing, message passing, token passing, network transmission, etc.

[0108] Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. If implemented by software, firmware, middleware, or microcode, program code or code segments for performing the required tasks may be stored on a machine-readable medium. A processor may perform the required tasks.

[0109] While the aforementioned specification describes aspects of the invention with respect to specific embodiments, those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects of the invention described above may be used individually or together. Furthermore, the embodiments may be used in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive.

[0110] In addition, for illustrative purposes, the method has been described in a specific order. It should be understood that in alternative embodiments, the method may be performed in a different order than that described. The method described above may be performed by hardware components, or it may be embodied in a sequence of machine-executable instructions that can be used to cause a machine, such as a general-purpose or dedicated processor, or an instruction-programmed logic circuit, to perform the method. It should also be understood that these machine-executable instructions may be stored on one or more machine-readable media, such as a CD-ROM or other type of optical disc, a floppy diskette, ROM, RAM, EPROM, EEPROM, magnetic card or optical card, flash memory, or other type of machine-readable media suitable for storing electronic instructions. Alternatively, the method may be carried out by a combination of hardware and software.

Claims

1. A computer-based method for managing runtime state and service configuration for containers in a multi-tenant environment, wherein the method is: The steps include receiving a first request for a first service provided by a first tenant, The steps include selecting a container in the aforementioned multi-tenant environment, The steps include loading a first configuration for realizing the first service into the container, The method further includes the step of loading runtime state into the container, wherein the runtime state includes the state of the first service when the first service previously served a previous request in a different container, and the method further includes A method comprising the step of servicing the first request from the container using the runtime state.

2. A step of determining that the first service has finished serving the first request, The steps include flushing the first configuration from the container, The method according to claim 1, further comprising the step of ensuring that the runtime state is stored in a data store outside the container for the subsequent execution of the first service.

3. The method according to claim 2, wherein the first configuration is provided by a tenant of the multi-tenant environment prior to runtime.

4. The method according to claim 2 or 3, wherein the data store includes a key-value data store.

5. The method according to claim 4, wherein the key-value data store includes a distributed key-value data store.

6. The method according to claim 4 or 5, wherein the identity of the tenant associated with the first service includes a key in the key-value data store, and the first configuration and runtime state include a value in the key-value data store.

7. The method according to any one of claims 2 to 6, wherein the data store also stores a registry of containers available in the multi-tenant environment.

8. The method according to claim 7, wherein when a new container becomes available in the multi-tenant environment, the datastore updates one or more routers in the multi-tenant environment based on the registry of the container.

9. The steps include receiving a second request for the first service after the first configuration has been flushed from the container, To serve the second request, the first configuration is loaded into a second container in the multi-tenant environment. The method according to any one of claims 1 to 8, further comprising the step of loading the runtime state into the second container.

10. A program that, when executed by one or more processors, includes instructions that cause one or more processors to perform an operation, wherein the operation is: Receiving a first request for a first service provided by a first tenant, Selecting containers in a multi-tenant environment, Loading the first configuration for realizing the first service into the container, The operation further includes loading runtime state into the container, wherein the runtime state includes the state of the first service when the first service previously served a previous request in a different container, and the operation further includes A program that includes servicing the first request from the container using the runtime state.

11. The aforementioned operation further, Assigning multiple containers to the first tenant in the multi-tenant environment, Identifying one or more containers in the plurality of containers that are assigned to the first tenant but are not used by the first tenant, The contents of one or more of the aforementioned containers are flushed, The program according to claim 10, further comprising reassigning one or more containers to a second tenant in the multi-tenant environment.

12. The program according to claim 11, wherein after the contents of one or more containers are flushed, the one or more containers are not assigned to any tenant for a first time interval before being reassigned to the second tenant.

13. The aforementioned operation further, Identifying one or more second containers in the plurality of containers that are assigned to the first tenant but are not used by the first tenant, It is determined that there are no other tenants who require one or more of the second containers mentioned above, The program according to claim 11 or 12, further comprising removing one or more second containers from the multi-tenant environment.

14. The program according to any one of claims 11 to 13, further comprising determining that the first tenant is receiving more requests than can be served by the plurality of containers.

15. The aforementioned operation further, Instantiating multiple new containers, The program according to claim 14, further comprising assigning the new plurality of containers to the first tenant.

16. The program according to claim 14, wherein the operation further includes assigning a container previously assigned to another tenant to the first tenant.

17. The program according to any one of claims 11 to 16, wherein the gateway of the multi-tenant environment reassigns one or more containers to the second tenant in the multi-tenant environment.

18. It is a system, One or more processors, The memory device includes one or more memory devices, and the memory device includes instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation, and the operation is Receiving a first request for a first service provided by a first tenant, Selecting containers in a multi-tenant environment, Loading the first configuration for realizing the first service into the container, The operation further includes loading runtime state into the container, wherein the runtime state includes the state of the first service when the first service previously served a previous request in a different container, and the operation further includes A system including servicing the first request from the container using the runtime state.

19. The system according to claim 18, wherein the container is one of a plurality of containers in the multi-tenant environment that are instantiated to serve requests from client devices.

Images