Service Orchestration in a Distributed Pod-Based System
Patent Information
- Application Number
- JP2023526941
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2020-11-10
- Filing Date
- 2021-11-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-11-08
AI Technical Summary
Existing resource management strategies for distributed computing environments, particularly in cloud computing, are inadequate for managing containerized systems across multiple environments, leading to challenges in scalability and operational costs, and pose unique privacy and security issues due to regulatory compliance in healthcare data handling.
A kernel-based system for resource allocation and isolation within distributed pod-based systems, utilizing customized approaches to provision and deploy services using containers wrapped in pods, with image templates hardened for security and optimized for performance, and integrated with Continuous Integration and Continuous Deployment (CI/CD) systems for regulatory compliance.
Enhances scalability and reduces operational costs by efficiently managing resources across multiple distributed computing environments while ensuring compliance with healthcare data privacy regulations, enabling secure and efficient deployment of services like machine learning models and medical devices.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical Field]
[0001] Priority claim This application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 111,996, filed November 10, 2020, which is incorporated herein by reference in its entirety for all purposes.
[0002] Field The present disclosure relates to digital personalized healthcare, and in particular to techniques for service orchestration within distributed pod-based systems. [Background technology]
[0003] background A cluster manager (also known as the kernel of a distributed computing system) provides resource sharing among multiple different frameworks (e.g., stream processing, batch processing, data integration, storage frameworks, etc.). In some examples, the cluster manager includes: (i) agents that execute tasks from the frameworks and notify them about available resources; (ii) frameworks for providing services or solving specific use cases; and (iii) a master that mediates between the agents and the frameworks. The frameworks include a scheduler that decides whether to accept or reject resources and an executor that assigns accepted resources to tasks and controls task execution. The resource sharing provided by the cluster manager generally includes resource allocation and resource isolation. Resource allocation includes agents that report available resources, a master that decides how many resources to provide each framework, and frameworks that decide which resources to accept and which computations to run on the resources. Resource isolation includes the agents, master, and frameworks ensuring that resources allocated to any task are not consumed by another task.
[0004] In healthcare, data-driven technological solutions are being developed to reduce costs and further personalize healthcare. As the healthcare landscape shifts to an on-demand deployment system of personalized medical services and solutions, healthcare providers are looking to developers for help innovating solutions more quickly by automating and streamlining software deployment and service management processes. To support healthcare providers and services, developers have turned to distributed computing environments (e.g., cloud computing) as a healthcare information technology infrastructure standard, offering a low-cost way to develop the complex infrastructure needed to support software deployment and service management processes within a service model (e.g., analytics as a service (AaaS)). While distributed computing environments such as cloud computing offer many benefits to healthcare providers, they function differently than legacy storage or information sharing solutions and therefore pose their own unique privacy and security challenges. For example, because users access data via an internet connection, compliance with government regulations (e.g., the Health Insurance Portability and Accountability Act (HIPAA), "good practice" quality guidelines and regulations (GxP), and the General Data Protection Regulation (GDPR) poses unique challenges for healthcare providers considering cloud solutions to support their software deployment and service management processes. Therefore, advances in compliant software deployment platforms built to ensure the confidentiality, availability, and integrity of protected healthcare information are needed. Summary of the Invention
[0005] overview In various embodiments, a computer-implemented method includes receiving, at a first kernel residing in a first distributed computing environment, a first request to initiate a deployment process for a first subservice or service on the first distributed computing environment, the first subservice or service having a type, and in order for the first subservice or service to be deployed on the first distributed computing environment, the type of the first subservice or service must be one that the first distributed computing environment is configured to support; and provisioning, by the first kernel, a specified resource in a deployment ring of the first distributed computing environment in response to receiving the request for the first subservice or service and the type that the first distributed computing environment is configured to support, wherein the provisioning includes obtaining an image based on the type of the first subservice or service, and a software package template including a specification of resources to be used to run the software package, the template being customized to use the specified resources based on a type of a first subservice or service; deploying the specified resources within a deployment ring; and using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a first pod for running one or more programs to provide the first subservice or service; and modifying the image using a writable layer to include source code necessary to run the one or more programs; and deploying, by a first kernel, the first subservice or service using one or more replicas of the first pod and the modified image.
[0006] In some embodiments, deploying the resources includes analyzing the image to identify specified resources; requesting available resources from the deployment ring; parsing the available resources from the deployment ring; determining whether all of the specified resources from the image are available in the deployment ring based on the parsing of the available resources; and, in response to all of the specified resources from the image not being available in the deployment ring, auto-scaling the available resources based on the image to provide additional resources to the deployment ring to satisfy the specified resources identified from the image.
[0007] In some embodiments, obtaining the image further includes identifying an image from the plurality of images based on a first sub-service or type of service, the image being configured in accordance with security best practices to reduce vulnerabilities of the image, the security best practices being selected based on the type of the first sub-service or service.
[0008] In some embodiments, the image is further configured with image specifications to optimize deployment workflow and performance, the image specifications being selected based on the type of the first sub-service or service.
[0009] In some embodiments, the first sub-service or type of service is machine learning model software, software as a medical device, software that operates in conjunction with a physical medical device, or data capture and processing software.
[0010] In some embodiments, the method further includes executing, by the kernel, the first subservice or service, wherein the executing includes: determining, by a master of the kernel, a framework for executing the first subservice or service based on a type of the first subservice or service; offering, by the master of the kernel, available resources in the placement ring to the framework for executing one or more programs to provide the first subservice or service; receiving, by the master of the kernel in response to the offer, information regarding tasks defined by the one or more programs; and sending, by the master of the kernel, the tasks to an agent of the kernel that allocates the specified resources to an execution unit of the framework for executing the tasks.
[0011] In some embodiments, the method further includes publishing, by the kernel, the first sub-service or service; and initiating, by the kernel, security for the first sub-service or service.
[0012] In some embodiments, the method further includes evaluating, by the kernel, functionality of the first subservice or service and performing a health check to determine whether the first subservice or service is functioning as expected; and if the first subservice or service fails the health check, notifying a continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected, wherein a first request is received from the CICD system; and when the first subservice or service is functioning as expected, (i) publishing, by the kernel, the first subservice or service; (ii) initiating, by the kernel, security for the first subservice or service; and (iii) notifying the CICD system regarding the first subservice or service successfully functioning as expected.
[0013] In some embodiments, the method includes receiving, at a second kernel residing in a second distributed computing environment, a second request to initiate a deployment process for a second sub-service of the service on the second distributed computing environment, the second sub-service having a type, and the type of the second sub-service or service must be one that the second distributed computing environment is configured to support in order for the second sub-service or service to be deployed on the second distributed computing environment; and provisioning, by the second kernel, a specified resource in a deployment ring of the second distributed computing environment in response to receiving the request, the provisioning including obtaining an image based on the type of the second sub-service or service, the image comprising the resource used to execute the software package. and deploying the second subservice or service using one or more replicas of the second pod and the modified image with a second kernel.
[0014] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.
[0015] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude any equivalents of the features shown and described, or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims. [Brief explanation of the drawings]
[0016] The present disclosure is described in conjunction with the accompanying drawings, in which:
[0017] [Figure 1] FIG. 1 illustrates a diagram of a digital health platform for providing data-driven technology solutions, according to various embodiments.
[0018] [Figure 2] 1 illustrates a diagram of a kernel in accordance with various embodiments.
[0019] [Figure 3] FIG. 1 illustrates a block diagram of a process for service orchestration in a distributed pod-based system, according to various embodiments.
[0020] [Figure 4] FIG. 1 illustrates a swimlane diagram illustrating a process for deploying a sub-service or service to a digital health platform using a kernel, according to various embodiments.
[0021] [Figure 5] 1 shows a flowchart illustrating a process for deploying one or more sub-services of a service on a software platform, according to various embodiments.
[0022] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the description is applicable to any of the similar components having the same first reference label, regardless of the second reference label. DETAILED DESCRIPTION OF THE INVENTION
[0023] Detailed Description I. Overview This disclosure describes techniques for service orchestration within a distributed pod-based system. More specifically, embodiments of the present disclosure provide techniques for provisioning and deploying at least some of the services (hereinafter referred to as sub-services) within the distributed pod-based system by a cluster manager (hereinafter referred to as a kernel) based on the type of sub-service or service.
[0024] A kernel provides resource allocation and isolation across applications or frameworks within a distributed computing environment (e.g., cloud computing). Cloud computing typically relies on virtualization, which involves provisioning cloud-based applications and services using virtual resources such as virtual machines. Efficient management of these resources is a challenge because it directly impacts both the scalability and operational costs of a distributed computing environment. One type of virtualization technology is containers, which provide resource portability and require minimal overhead compared to traditional virtual machines. However, because traditional resource management strategies are designed for allocating and migrating virtual machines, the challenge arises of how these strategies can be adapted to manage a containerized distributed computing environment. Separately, cloud computing solutions such as AaaS and Software as a Service (SaaS) are also no longer limited to a single distributed computing environment. In some cases, services are provisioned and deployed across multiple distributed computing environments (i.e., multi-cloud solutions that use multiple cloud providers and their provided computer hardware and physical network infrastructure to meet various technical or business requirements), allowing organizations to build portable software stacks that offer a greater set of capabilities than can be obtained from a single distributed computing environment, are free of vendor lock-in, and are DevOP-driven. However, because traditional resource management strategies are designed for allocating and migrating virtual machines within a single distributed computing environment, an additional challenge arises: how can these strategies be adapted to manage multiple container distributed computing environments?
[0025] To address these limitations and challenges, the disclosed technique for service orchestration within a distributed pod-based system utilizes a customized approach to resource allocation and isolation. This technique aims to provision resources for subservices or services deployed in a distributed computing environment. The deployed subservices or services have a type, and the type of subservice or service deployed on a given distributed computing environment must be one that the distributed computing environment is configured to support. The subservice or service type can be model software, such as a machine learning model, a regression model, or any other type of supervised or unsupervised model; machine learning model software; software as a medical device (SAMD); software operating in conjunction with a physical medical device; or data ingestion and processing software. Furthermore, the subservices or services are deployed using one or more containers wrapped into a pod. The one or more containers are configured using an image obtained based on the type of the subservice or service deployed within the one or more containers. Each image is a binary that contains all the requirements for running a single container as well as metadata describing the container's needs and capabilities.
[0026] One exemplary embodiment of the present disclosure relates to a method that includes receiving, at a first kernel residing in a first distributed computing environment, a first request to initiate a deployment process for a first subservice or service on the first distributed computing environment, the first subservice or service having a type, and the type of the first subservice or service must be one that the first distributed computing environment is configured to support in order for the first subservice or service to be deployed on the first distributed computing environment. The method further includes provisioning, by the first kernel, specified resources in a deployment ring of the first distributed computing environment in response to receiving the request for the first subservice or service and the type that the first distributed computing environment is configured to support. The provisioning includes obtaining an image based on a first subservice or service type, the image being a software package template including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the first subservice or service, deploying the specified resources within a deployment ring, using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a first pod for running one or more programs to provide the first subservice or service, and modifying the image using a writable layer to include source code necessary to run the one or more programs. The method further includes deploying, by a first kernel, the first subservice or service using one or more replicas of the first pod and the modified image.
[0027] In some cases, the method further includes receiving, at a second kernel residing in the second distributed computing environment, a second request to initiate a deployment process for a second subservice of the service on the second distributed computing environment. The second subservice or service has a type, and in order for the second subservice or service to be deployed on the second distributed computing environment, the type of the second subservice or service must be one that the second distributed computing environment is configured to support. The method further includes provisioning, by the second kernel, the specified resource in the deployment ring of the second distributed computing environment in response to receiving the request and the type of the second subservice or service that the second distributed computing environment is configured to support. The provisioning includes obtaining an image based on a second subservice or service type, where the image is a software package template including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the second subservice or service type, deploying the specified resources within a deployment ring, building one or more containers using the image that encapsulate the specified resources, where the one or more containers are wrapped into a second pod for running one or more programs to provide the second subservice or service, and modifying the image using a writable layer to include source code necessary to run the one or more programs. The method further includes deploying the second subservice or service using one or more replicas of the second pod and the modified image with a second kernel.
[0028] II. Digital Health Platform FIG. 1 shows a simplified diagram of a digital health platform 100 for providing data-driven technology solutions according to various embodiments. In the illustrated embodiment, the digital health platform 100 includes client computing devices 105 coupled to a cloud-based infrastructure 110 via a network 115 including a network gateway 120 and a network mesh 125. The infrastructure 110 is adapted to run services or software applications within service pods 130 using resources provisioned within a deployment ring 135 by a cloud service provider 140 (e.g., a distributed computing environment) using various hardware and cloud infrastructures (e.g., private or on-premise cloud infrastructures and public cloud infrastructures). These services or software applications may be provided to users of the client computing devices 105 as web-based or cloud services, e.g., under an AaaS or SaaS model. Several providers offer cloud services, such as Amazon, Google, and Oracle. The term cloud service is generally used to refer to services made available to users on demand over a communications network, such as the Internet, by a service provider's system (e.g., infrastructure 110), such as a government-regulated entity. Thus, consumers may utilize cloud services offered by service providers without having to purchase separate licenses, support, or hardware and software resources to support the services. For example, a cloud service provider's system may host one or more programs, and users may use the one or more programs, over the Internet, on-demand, without the users having to purchase infrastructure resources to run the one or more programs.Cloud services are designed to provide easy and scalable access to applications, resources, and services.
[0029] In some cases, a user (e.g., a software or service consumer) operating a client computing device 105 utilizes one or more client applications to consume software products, services, or systems provided by the various components 145 of the infrastructure 110. In other examples, a user (e.g., a developer) operating a client computing device 105 utilizes one or more client applications to upload source code for software products, services, or systems provided by the various components 145 of the infrastructure 110. The components 145 include software components that may be executed by one or more processors, hardware components, or combinations thereof. It should be understood that a variety of different system configurations are possible, which may differ from that depicted for the digital health platform 100. Thus, the embodiment depicted in FIG. 1 is an example of a distributed computing environment for implementing a digital health platform and is not intended to be limiting.
[0030] The client computing devices 105 include various types of computing systems, such as portable handheld devices, general-purpose computers such as personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computing devices may run a variety of mobile operating systems (e.g., Microsoft Windows Mobile (登録商標)Various types and versions of software applications and operating systems (e.g., Microsoft Windows, iOS, Windows Phone, Android, BlackBerry, Palm OS) are supported. (登録商標) , Apple Macintosh (登録商標) , UNIX (登録商標) or a UNIX-based operating system, Linux or a Linux-based operating system such as Google Chrome™ OS. Portable handheld devices include mobile phones, smartphones, (e.g., iPhones, (登録商標) ), tablets (e.g., iPad (登録商標) ), personal digital assistants (PDAs), etc. Wearable devices include Fitbit Versa (商標) Smartwatch, Magic Leap 1 (登録商標) , HTV Vive, and Oculus (登録商標) Gaming systems may include virtual reality (VR) or augmented reality (AR) systems such as the Kinect VR, VR- ... (登録商標) Microsoft Xbox with or without gesture input device (登録商標) game consoles, Sony PlayStation® systems, various game systems offered by Nintendo®, and others. Client device 105 may be capable of running a variety of different applications, such as various internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), and may use a variety of communication protocols.
[0031] Network 115 may be any type of network familiar to those skilled in the art that is capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk®, etc. By way of example only, network 115 may be a local area network (LAN), Ethernet, token ring, wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 suite of protocols, Bluetooth®, etc. (登録商標) , and / or any other wireless protocol), and / or any combination of these and / or other networks.
[0032] The network gateway 120 is a network node that forms a secure pathway between two or more of the networks 115 that operate on the same or different protocols. The network gateway 120 may provide network security using one or more of the following technologies: firewalls to monitor incoming and outgoing network traffic, virtual private networks to provide private, secure communication channels, security scans to identify security flaws in the network, access managers for authentication and authorization services, etc. The network gateway 120 routes network traffic using routers and service connectors that manage access to various software products, services, or systems (e.g., using a service subscription business model). The network mesh 125 is a local network topology in which the infrastructure 110 (e.g., bridges, switches, and other infrastructure devices) directly, dynamically, and non-hierarchically connect to as many other nodes as possible and cooperate with each other to efficiently route data between the devices and nodes. The network mesh 125 manages connectivity using one or more of techniques such as load balancing, product, service, or system discovery, network access, routing, and peering, traffic mirroring, etc. The network 115 , network gateway 120 , and network mesh 125 work in combination to manage all data flowing in and out of the infrastructure 110 .
[0033] Components 145 include one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, application-specific servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination of computers or systems operating individually or in combination to provide resources, data, services, or programs to client computing devices 105 over network 115. Components 145 may further include other computing architectures that involve virtualization, such as one or more virtual machines running virtual operating systems, or one or more flexible pools of logical storage that can be virtualized to maintain virtual storage. In various embodiments, components 145 are adapted to run one or more services or software applications that provide the functionality described in this disclosure.
[0034] Component 145 also includes one or more data repositories. These data repositories may, in various embodiments, be used to store data and other information. For example, one or more of the data repositories may be used to store information for providing a data-driven technology solution, such as a SAMD, and to store information for validating and deploying source code to implement the data-driven technology solution. The data repositories may reside in various locations. For example, a data repository used by a component may be local to the component or may be remote from the component and communicate with the component via a network-based or dedicated connection. The data repositories may be of different types. In particular embodiments, the data repository used by the component may be a database, such as a centralized database, a distributed database, a NoSQL database, a relational database, or the like. One or more of these databases may be adapted to enable storage, update, and retrieval of data to and from the database in response to SQL-formatted commands. In particular embodiments, one or more of the data repositories may also be used by an application to store application data. The data repository used by an application may be of different types, such as a key-value store repository, an object store repository, or a general storage repository supported by a file system.
[0035] Components 145 also include computing nodes adapted to run one or more programs, such as services or software applications that provide the functionality described in this disclosure (e.g., services or software applications offered as web-based or cloud services, or applications for implementing a continuous integration and continuous deployment (CI / CD) system). Each node is a representation of a single machine, optionally implemented within a cluster of nodes. A single machine may be a physical machine (e.g., a server in a data center) or a service provider (e.g., a cloud service provider) with a set of available CPU and RAM resources. (商標) The infrastructure 110 may be a virtual machine hosted on a cloud provider such as AWS. In a cluster, nodes pool their resources to form a more powerful machine. When one or more programs are deployed on the cluster, the cluster intelligently handles distributing work to the individual nodes. As nodes are added or removed, the cluster can shift work as needed. Which individual machine actually runs the code is not important to the program(s) or to the infrastructure 110.
[0036] One or more programs deployed to one or more clusters are packaged as containers. Containers are a widely accepted standard, and various images can be defined for deploying one or more programs on the infrastructure 110. Containerization enables the infrastructure 110 to create self-contained execution environments. Any program and all of its dependencies can be packaged into a single file and then shared across the infrastructure 110. Container creation can be done programmatically, enabling a powerful, fully automated CI / CD pipeline used to validate and deploy code on the infrastructure 110. Containers are wrapped into higher-level constructs known as pods 130. Containers within the same pod 130 can share the same resources and local network. In some cases, containers can communicate with other containers within the same pod 130 as if they were on the same machine, while maintaining a degree of isolation from others. Pods 130 are used as the unit of replication within the infrastructure 110. When a program or resource becomes overwhelmed with processing and a single pod 130 instance cannot carry the load, the infrastructure 110 can be configured to deploy new replicas of the pod 130 in a cluster as needed. Even when not under heavy load, it can be beneficial to have multiple copies of the pod 130 running at any time in a production system to enable load balancing and fault tolerance. One or more instances of the pod 130 are provisioned in a cloud infrastructure system provided by one or more cloud service providers 140.
[0037] The cloud infrastructure system provided by one or more cloud service providers 140 includes infrastructure resources utilized to facilitate the provisioning of one or more instances of pods 130 that support various cloud services provided by infrastructure 110. To facilitate efficient utilization of these resources for provisioning one or more instances of pods 130, the resources may be bundled into sets of resources or resource modules (also referred to as “deployment rings 135”). Each resource module or deployment ring 135 may include a pre-integrated and optimized combination of one or more types of resources. In particular examples, different deployment rings 135 may be pre-provisioned for different types of cloud services. For example, a first set of deployment rings 135 may be provisioned for SAMD services, a second set of deployment rings 135 may include a different combination of resources than the deployment rings 135 in the first set of deployment rings 135, may be provisioned for data analytics services, and so on. For some cloud services, resources allocated to provisioning services may be shared between services.
[0038] The digital health platform 100 further includes one or more kernels 150. The kernels 150 are adapted to run on each cloud infrastructure system (e.g., a distributed computing environment) provided by one or more cloud service providers 140. The kernels 150 are cluster managers that provide resource allocation and isolation across distributed applications or frameworks across the digital health platform 100. The kernels 150 provide an application programming interface (API) to one or more programs for orchestration of services and software, including resource management and scheduling. The architecture of the kernels 150 includes agent nodes for executing tasks, master nodes for sending tasks to agent nodes, a domain manager for election and for looking up addresses of master nodes, and a framework for coordinating with master nodes to schedule tasks on agent nodes.
[0039] The digital health platform 100 further includes a CI / CD system 155. The CI / CD system 155 is implemented within a cloud infrastructure system, allowing the digital health platform 100 to frequently update, test, and distribute changes within the source code of a software product, service, or system. As described in detail herein, in healthcare, there are government regulations regarding data security (e.g., data integrity and data privacy) that software must comply with. In the CI / CD system 155, these policy regulations can be included in the code, allowing compliance to be automatically tracked, verified, and reconfigured. In the SAMD example, data storage locations, server access controls, and activity logging can be included in the source code to protect and manage user data during software use. Encryption and password-protected operations can further be included during continuous integration. During continuous distribution, security and monitoring tools can be used to track user activity and detect errors that may lead to security threats.
[0040] The CI / CD system 155 may also be used to provision machine learning models. Machine-learning models are initially trained using a dataset, but over time, the model may drift or the data may change, resulting in the need for an updated machine learning model. When a machine learning model runs within a software application, code associated with the software application may include triggers for when the machine learning model should be retrained. For example, the code may include instructions to retrain the machine learning model at predetermined time intervals, when new training data is available, or when the machine learning model's performance is determined to fall below a threshold. Additionally, software developers may explore variations in model architecture and hyperparameters in a test environment based on monitoring the performance of the machine learning model in a production environment or based on estimated improvements for model optimization. The CI / CD system 155 enables machine learning models to be easily built, tested, and deployed to a production environment when they are determined to meet performance requirements.
[0041] III. Kernel FIG. 2 illustrates a simplified diagram of a kernel 200 (e.g., kernel 150 described with respect to FIG. 1 ) for resource allocation and isolation across a distributed application or framework, according to various embodiments. In the illustrated embodiment, kernel 200 includes framework software logic, including master 205, backup master 210, domain manager 215, agents 220(a-n), and scheduler 225 and executor 230. Kernel 200 groups and provisions resources 235 within a container-based environment based on the type of subservice or service being deployed in the distributed computing environment. In particular, the subservice or service type must be one that the distributed computing environment is configured to support. The subservice or service type can be model software, such as a machine learning model, a regression model, or any other type of supervised or unsupervised model; machine learning model software; a self-assessment model (SAMD); software operating in conjunction with a physical medical device; or data acquisition and processing software. This type restriction supports efficient and effective management of resources 235, which has a direct positive impact on both the scalability and operational costs of the distributed computing environment. Using basic scripts or complex "orchestrators," developers can quickly recover crashed subservices or service components, add new instances to meet increased demand, or perform rolling upgrades to update subservices or services without any downtime. Additionally, container-based environments are more lightweight than virtual machines and can share a single operating system, significantly reducing infrastructure costs.
[0042] The kernel 200 groups the resources 235 of machines / nodes located within a cluster or deployment ring together into one or more containers 240(a-n) wrapped into a single pod instance 245(a-n). From the single pod instance 245(a-n), various tasks 250 for one or more programs can be executed, utilizing the resources 235 to provide subservices or services. One or more containers 240(a-n) are configured using images derived based on the type of subservice or service deployed within the container or containers. Each image is a binary that includes all the requirements for running a single container, as well as metadata describing the container's needs and capabilities. Containers enable developers to create consistent, reproducible environments that are isolated from each other and can contain dependencies. While isolation is not perfect (e.g., authorized services may communicate with each other via API calls), containers isolate critical resources, such as container access to underlying CPU, memory, storage, and network resources, between each container. This reduces the possibility of individual containers consuming excessive resources and also prevents potential security issues.
[0043] The domain manager 215 is a centralized configuration manager used by the kernel 200 to coordinate activities across a cluster or deployment ring. The domain manager 215 can select a lead master 205, select a backup master 210 to replace a failed lead master 205, and provide fault tolerance by selecting agents 220 to join a cluster or deployment ring. The master 205 manages the agents 220 running on each cluster node and the frameworks that execute tasks on the agents 220. The master 205 implements the provisioning of specific resources within a cluster or deployment ring and fine-grained sharing between frameworks using resource offerings. Each resource offering is a list of available resources 235 available to multiple agents 220. The master 205 determines the number of resources 235 to provide to each framework according to organizational policies such as fair sharing or priority. To support a diverse set of inter-framework allocation policies, the kernel 200 is adapted to allow users to define their own policies via pluggable allocation modules.
[0044] Each framework running on the kernel 200 consists of two components: a scheduler 225 that registers offered resources 235 with the master 205, and an executor 230 that is launched on the agent 220 to execute the framework's tasks 250. The master 205 determines the number of resources 235 to offer to each framework, but the framework's scheduler 225 selects which of the offered resources 235 to use. Once the framework accepts the offered resources 235, it passes to the master 205 a description of one or more programmatically defined tasks 250 that the framework wants to execute on the agent 220. For example, the framework may schedule one or more tasks to execute as follows: First, the agent 220 reports to the master that it has resources 235, such as six CPUs and 9 GB of free memory. The master 205 then invokes an allocation module that informs the master 205 that the framework should be provided with all available resources. The master 205 then sends a resource offer to the framework describing all available resources 235. The framework's scheduler 225 responds to the master 205 with information about one or more programmatically defined tasks 250, e.g., two tasks to be executed on the agent, using three CPUs and 1 GB of RAM for the first task, and two CPUs and 6 GB of RAM for the second task. Finally, the master 205 sends the tasks 250 to the agent 220, which allocates the appropriate resources 235 to the framework's execution unit 230 and launches the two tasks. Because one CPU and 2 GB of RAM are still free, the allocation module can provide them to another framework. This resource provisioning process is repeated as tasks finish and new resources become available.
[0045] IV. Technology for deploying services on the digital health platform 3-5 illustrate processes and operations for service orchestration within a distributed pod-based system. Individual embodiments may be described as a process depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process terminates when its operations are completed but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to a calling function or a main function.
[0046] The processes and / or operations illustrated in FIGS. 3-5 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processor cores), hardware, or a combination thereof. The software may be stored in memory (e.g., on a memory device, on a non-transitory computer-readable storage medium). The particular sequence of process steps illustrated in FIGS. 3-5 is not intended to be limiting. Other sequences of steps may be performed according to alternative embodiments. For example, in alternative embodiments, the steps outlined above may be performed in a different order. Furthermore, individual steps illustrated in FIGS. 3-5 may include multiple sub-steps, which may be performed in various sequences as appropriate for the individual steps. Furthermore, additional steps may be added or deleted depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.
[0047] 3 is a high-level block diagram of a process 300 for service orchestration in a distributed pod-based system in accordance with various aspects of the present disclosure. The process illustrated in diagram 300 is implemented by the architecture, systems, and techniques illustrated in FIGS. 1 and 2.
[0048] In block 305, image templates are configured for various types of subservices or services to be deployed. The image templates are configured using a script of instructions that defines how to build a particular image template. The script of instructions may be written based on service hardware requirements by a user, such as an administrator of a software deployment service or a software developer deploying a given type of subservice or service. Subservice or service types include, but are not limited to, model software such as machine learning models, regression models, or any other type of supervised or unsupervised model; SAMD; software operating in conjunction with physical medical devices; and data ingestion and processing software (e.g., stream processing, batch processing, long-running services, data analysis and processing, data storage, etc.). Furthermore, these categories are not mutually exclusive; for example, a SAMD application may include aspects of machine learning models or data ingestion and processing. Each image template is a binary that contains all the requirements for running a single container for a certain type of subservice or service, as well as metadata that describes the container's needs and capabilities. The template image is used to build containers for running one or more programs to provide the subservice or service type. Thus, each container is essentially an execution image of one or more programs. However, the image template from which a container is built exists separately and cannot be changed. When a container is built using an image template, a read-and-write copy of the requirements for running the container (e.g., resources specified for a subservice or service) is created within the container. When a container is built, it adds a writable layer on top of the immutable template image, which means that the template image can be modified to include the source code, libraries, dependencies, tools, and other files necessary to run one or more programs to provide a subservice or service.An unlimited number of images for running one or more programs can be created from a single image template.
[0049] In block 310, the image template is hardened based on the type of subservice or service for which the image template is configured for deployment. Hardening is the process of configuring an image according to security best practices to reduce the image's vulnerabilities (e.g., hardening software packages to reduce exposure surfaces and attack vectors using different tools and configurations). Security best practices are cloud service provider agnostic and configurable independently for the type of subservice or service for which the image template is configured for deployment. For example, a first set of security best practices may be defined for hardening an image template configured for data analysis and processing. The first set of security best practices takes into account weaknesses commonly found in data analysis and processing. Furthermore, a second set of security best practices may be defined for hardening an image template configured for SAMD. The second set of security best practices takes into account weaknesses commonly found in SAMD. Furthermore, a third set of security best practices may be defined for hardening an image template configured for artificial intelligence or machine learning processing. The third set of security best practices takes into account weaknesses commonly found in artificial intelligence or machine learning processing. Although only three types of hardening processes are shown in FIG. 3, it should be understood that multiple additional or alternative hardening processes may be available for hardening image templates, including stream process hardening and batch process hardening.
[0050] In block 315, the hardened image template is tuned based on the type of subservice or service for which the image template is configured for deployment. Tuning is the process of configuring the image according to image specifications to optimize deployment workflow and performance (e.g., improving resource usage such as execution time, memory usage, disk space usage, network usage, etc.). The image specifications are cloud service provider agnostic and configurable independently for the type of subservice or service for which the image template is configured for deployment. For example, a first set of image specifications may be defined to tune an image template configured for stream processing. The first set of image specifications considers behaviors and issues commonly observed with stream processing. Furthermore, a second set of image specifications may be defined to tune an image template configured for input / output processing. The second set of image specifications considers behaviors and issues commonly observed with input / output processing. Furthermore, a third set of image specifications may be defined to tune an image template configured for network processing. The third set of image specifications considers behaviors and issues commonly observed with network processing. Although only three types of adjustment processes are shown in FIG. 3, it should be understood that multiple additional or alternative adjustment processes may be available for adjusting image templates, including batch process adjustments and long-running service adjustments.
[0051] At block 320, the (hardened and / or adjusted) image template is registered with a registry. The registry provides services for storing and retrieving images for one or more repositories (i.e., image artifact factories). Each repository contains one or more image templates (hardened and / or adjusted).
[0052] In block 325, the actors release the source code for review, validation, and deployment to a CI / CD system for the software platform (e.g., the government-regulated entity's digital health platform 100 described with reference to FIG. 1). In various cases (e.g., when the actors are third-party developers or partners), the source code has already been validated according to a quality management system (QMS) associated with each actor. For example, each actor may establish a software lifecycle model for validating the developed software within the framework of a QMS appropriate for its product and organization. A QMS is a set of interrelated or interacting elements, such as policies, goals, procedures, processes, and resources, established individually or collectively to guide an organization. For each software lifecycle activity, there are executable tasks that support the conclusion that the software is validated. However, the tasks to be performed, the order in which they are performed, and the recurrence and timing of their execution are determined by the specific software lifecycle model selected and the safety risks associated with the software application as individually perceived by the actors. Each task for validating software typically begins with requirements, for example, for each function of the software, and the actors must be able to point to requirements that describe the function. Additionally, for each requirement, the actor should have a plan for testing that functionality to ensure it can function as needed, and a record-keeping system for recording evidence that the plan was executed and the results of the testing. Once the source code has been verified and validated according to the actor's QMS, the actor may release the source code to the CI / CD system for verification, validation, and deployment according to the software platform's QMS.
[0053] In block 330, the software platform's CI / CD system receives the source code and automatically performs quality and compliance checks on the source code and executable program in accordance with the QMS associated with the software platform. A determination is then made as to whether the source code and executable program are valid and / or contain open source vulnerabilities. If the source code and executable program are invalid (do not meet all or part of the requirements of the QMS) and / or contain open source vulnerabilities (as defined and determined by the software platform in accordance with the QMS), the CI / CD system forwards the source code back to the actor to fix the defects and / or vulnerabilities in the source code. In contrast, if the source code and executable program are valid (meet all or part of the requirements of the QMS) and / or do not contain open source vulnerabilities (as defined and determined by the software platform in accordance with the QMS), the CI / CD system notifies the kernel to begin the deployment process for the subservice or service using the source code or software.
[0054] In block 335, a kernel (e.g., kernel 200 described with respect to FIG. 2) receives a request to initiate a deployment process for a subservice or service using source code or software. The kernel is deployed in a distributed computing environment.
[0055] In block 340, the kernel parses the request to identify the type of subservice or service to be deployed, as well as the source code, libraries, dependencies, tools, and other files required to execute one or more programs to provide the subservice or service. The subservice or service type may include, but is not limited to, model software such as a machine learning model, a regression model, or any other type of supervised or unsupervised model, a SAMD, software operating in conjunction with a physical medical device, or data ingestion and processing software (e.g., stream processing, batch processing, long-running services, data analysis and processing, data storage, etc.). In order for a subservice or service to be deployed in a distributed computing environment, the subservice or service type must be one that the distributed computing environment is configured to support. The kernel checks the type of the subservice or service to be deployed against the subservice or service types supported by the distributed computing environment. If the subservice or service type is not one that the distributed computing environment is configured to support, the kernel forwards the request to a different kernel on a different distributed computing environment that may be configured to support the subservice or service type. Thus, the distributed computing environment that a kernel ultimately identifies as configured to support that subservice or service type may be the same as or different from the distributed computing environment running the kernel that initially receives the request for the subservice or service.
[0056] In block 345, if the subservice or service type is one that the distributed computing environment is configured to support, the kernel retrieves an image for provisioning the specified resources in the deployment ring of the distributed computing environment. The image is retrieved from one or more repositories using the registry described with respect to block 320. The image is a software package template that includes specifications for the resources used to run the software package, and the template is customized to use the resources based on the type of the first portion of the service being deployed. For example, if a SAMD subservice is being deployed, the kernel retrieves an image template configured for SAMD processing.
[0057] The kernel requests available resources in the placement ring in block 350. The available resources in the placement ring are parsed and a determination is made as to whether all of the specified resources from the image are available in the placement ring based on the parsing of the available resources.
[0058] In block 355, if all of the specified resources from the image are available in the placement ring, the kernel initiates a build and deploy process using the image to build one or more containers that encapsulate the specified resources. If all of the specified resources from the image are not available in the placement ring, the kernel initiates an auto-scaling process to scale up available resources based on the image to provide additional resources to the placement ring to meet the specified resources identified from the image. The kernel then initiates a build and deploy process using the image to build one or more containers that encapsulate the available resources and the additional resources. Once the one or more containers are built, the one or more containers add a writable layer on top of the image and modify the image to include source code, libraries, dependencies, tools, and other files necessary to run one or more programs to provide the subservice or service. The one or more containers are wrapped into a single pod instance to run one or more programs to provide the subservice or service. The subservice or service is deployed using the single pod instance and one or more replicas of the modified image.
[0059] At block 360, the sub-service or service is published and security for the sub-service or service is initiated, including (i) authentication / authorization processes for identity and access management, and (ii) monitoring of authentication / authorization processes and use of the sub-service or service for vulnerability concerns.
[0060] 4 illustrates a process 400 for deploying a subservice or service on a software platform using a kernel. The process illustrated in flowchart 400 is implemented by the architecture, systems, and techniques illustrated in FIGS. 1 and 2.
[0061] In step 405, the software development system's CI obtains the source code for deployment and executes a build process to build an executable program. As part of the build process, the software development system's CI tests and verifies the source code and executable program according to the software development system's QMS. The QMS defines a set of requirements for validating source code appropriate to the release manager's product and organization. For example, a set of requirements may be defined to determine whether the source code fits its intended use, functions as intended to implement its intended use, and meets a base level of security. As an additional part of the build process, the software development system's CI checks the quality of the source code. Code quality checks include checks for reliability (e.g., determining whether the code functions as intended with minimal defects), maintainability (e.g., follows a consistent structure / style, is easy to understand, is documented, etc.), and inspectability (e.g., the extent to which the software supports testing efforts for verification / validation). A determination is then made as to whether the source code and executable program are valid and / or of sufficient quality. If the source code and executable programs are invalid (do not meet all or part of the requirements of the QMS) and / or the quality of the source code and executable programs is low (as defined and measured by the software development system in accordance with the QMS), the CI of the software development system will transfer the source code back to the development team to fix the source code defects and / or quality. In contrast, if the source code and executable programs are valid (meet all or part of the requirements of the QMS) and / or the source code and executable programs are of good quality (as defined and measured by the software development system in accordance with the QMS), the CI of the software development system will suspend the release of the source code into the software platform.
[0062] The software platform's CI / CD system receives the source code in step 410. In some cases, the entity that controls the CI / CD system is the same as or different from the entity that controls the software development system.
[0063] In step 415, the CI / CD system executes a build process to build an executable program based on the received source code. As part of the build process, the CI / CD system tests and validates the source code and executable program according to the software platform's QMS. The QMS defines a set of requirements for validating source code appropriate for the software platform. For example, the set of requirements may be defined to determine whether the source code functions as intended to implement the intended use, meets an extended level of security, satisfies requirements enforced by international, national, and / or local regulations, satisfies data privacy concerns, and meets performance requirements specific to the container and software platform environment. As an additional part of the build process, the CI / CD system white-sources the source code. White-sourcing includes identifying open-source features in the source code, determining any permissions or licenses associated with the open-source features, and compiling the permissions or licenses and associated open-source features into a data structure. A determination is then made whether the source code and executable program are valid and / or contain open-source vulnerabilities. If the source code and executable programs are invalid (do not meet all or part of the requirements of the QMS) and / or contain vulnerabilities due to open source (as defined and determined by the software platform in accordance with the QMS), the CI / CD system transfers the source code back to the development team to fix the defects and / or vulnerabilities in the source code. In contrast, if the source code and executable programs are valid (meet all or part of the requirements of the QMS) and / or do not contain vulnerabilities due to open source (as defined and determined by the software platform in accordance with the QMS), the CI / CD system proceeds with the source code or software deployment process.
[0064] In step 420, a kernel (e.g., kernel 200 described with respect to FIG. 2) receives a request to initiate a deployment process for a subservice or service using source code or software. The kernel is deployed in a distributed computing environment (which is part of a software platform).
[0065] In step 425, the kernel parses the request to identify the type of subservice or service to be deployed, as well as the source code, libraries, dependencies, tools, and other files required to execute one or more programs to provide the subservice or service. The subservice or service type may include, but is not limited to, model software such as a machine learning model, a regression model, or any other type of supervised or unsupervised model, a SAMD, software operating in conjunction with a physical medical device, or data ingestion and processing software (e.g., stream processing, batch processing, long-running services, data analysis and processing, data storage, etc.). In order for a subservice or service to be deployed in a distributed computing environment, the subservice or service type must be one that the distributed computing environment is configured to support. The kernel checks the type of the subservice or service to be deployed against the subservice or service types supported by the distributed computing environment. If the subservice or service type is not one that the distributed computing environment is configured to support, the kernel forwards the request to a different kernel on a different distributed computing environment that may be configured to support that subservice or service type.
[0066] In step 430, if the subservice or service type is one that the distributed computing environment is configured to support, the kernel requests available resources from a placement constraint process running across one or more placement rings in the distributed computing environment. Each placement ring may include a pre-integrated and optimized combination of one or more types of resources. In certain examples, different placement rings may be pre-provisioned for different types of cloud services. For example, a first set of placement rings may be provisioned for SAMD services, while a second set of placement rings may include a different combination of resources than the first set of placement rings and may be provisioned for data analytics services, etc.
[0067] In step 435, the placement constraint process parses available resources on one or more placement rings in the distributed computing environment. The parsing is performed to obtain a list of all available resources on one or more placement rings in the distributed computing environment.
[0068] In step 440, the placement constraint process makes a determination as to whether all of the specified resources from the image are available within the placement rings based on a parsing of the available resources. The image is retrieved from an image artifact factory (one or more repositories) based on the type of subservice or service. The image is a software package template that includes specifications for resources used to execute the software package, and the template is customized to use the specified resources based on the type of subservice or service. When all of the specified resources from the image are available within one or more placement rings, the placement constraint process reserves the specified resources and notifies the kernel that the specified resources are available and reserved. If all of the specified resources from the image are not available within the placement rings, the placement constraint process auto-scales the available resources based on the image to provide additional resources to one or more placement rings to meet the specified resources identified from the image. The placement constraint process then reserves the specified resources and notifies the kernel that the specified resources are available and reserved.
[0069] In step 445, the kernel requests deployment of the subservice or service via the orchestration engine. The request includes information about the location of the specified resource within one or more deployment rings, an image associated with the type of subservice or service to be deployed, and the source code, libraries, dependencies, tools, and other files required to run one or more programs to provide the subservice or service.
[0070] In step 450, the orchestration engine detects the type of orchestration to be executed for deployment of the subservice or service, which includes identifying the type of orchestration from multiple types of orchestrations based on the type of the subservice or service.
[0071] In step 455, the orchestration engine uses the image to build one or more containers that encapsulate the resources specified based on the orchestration type for deployment of the subservice or service. Once the container or containers are built, the container or containers add a writable layer on top of the image, modifying the image to include the source code, libraries, dependencies, tools, and other files necessary to run the one or more programs to provide the subservice or service. The one or more containers are wrapped into a single pod instance for running the one or more programs to provide the subservice or service. The subservice or service is deployed using the single pod instance and one or more replicas of the modified image. The orchestration engine then notifies the kernel about the success or failure of the deployment of the subservice or service (and the reason for the failure).
[0072] In step 460, if the deployment is unsuccessful, the kernel notifies the CI / CD system so that corrective action can be taken. If the deployment is successful, the kernel performs a health check to evaluate the functionality of the subservice or service to determine whether the subservice or service is functioning as expected. If the service is not functioning as expected, the kernel notifies the CI / CD system that the service is not functioning as expected.
[0073] In step 465, when the service is functioning as expected, the kernel (i) publishes the subservice or service (e.g., registers the subservice or service for use by one or more subscribers or users), (ii) initiates security for the subservice or service, and (iii) notifies the CICD system regarding the success of the service to function as expected. Security includes (i) authentication / authorization processes for identity and access management, and (ii) monitoring the authentication / authorization processes and use of the subservice or service for vulnerability concerns.
[0074] FIG. 5 illustrates a process 500 for deploying one or more subservices of a service on a software platform. In step 505, a first request is received at a first kernel residing in a first distributed computing environment. The first request is to initiate a deployment process for a first subservice of the service on the first distributed computing environment. The first subservice has a type, and in order for the first subservice to be deployed on the first distributed computing environment, the type of the first subservice must be one that the first distributed computing environment is configured to support. By configured to support, it is meant that the distributed computing environment provides the computing power, memory storage, content delivery, and other functionality to reliably execute and scale the type of subservice. In some cases, the type of the first subservice may include, but is not limited to, model software such as a machine learning model, a regression model, or any other type of supervised or unsupervised model; software as a medical device; software operating in conjunction with a physical medical device; or data acquisition and processing software.
[0075] To ensure that the type of the first subservice is one that the first distributed computing environment is configured to support, the entity sending the request (e.g., a CI / CD system) determines the type of the first subservice, determines the types of subservices that the multiple distributed computing environments are configured to support, selects a given distributed computing environment (e.g., the first distributed computing environment) that is configured to support the type of the first subservice, and sends the request to the given distributed computing environment (e.g., the first distributed computing environment). Alternatively, to ensure that the type of the first subservice is one that the first distributed computing environment is configured to support, the entity sending the request (e.g., a CI / CD system) determines the type of the first subservice and sends a first request having the type of the first subservice. The first kernel checks the type of the first subservice to be deployed against the types of subservices supported by the first distributed computing environment, and if the type of the first subservice is not one that the first distributed computing environment is configured to support, the first kernel forwards the request to a different kernel on a different distributed computing environment that may be configured to support the type of the first subservice. In contrast, if the type of the first subservice is one that the first distributed computing environment is configured to support, the process continues at step 510.
[0076] In step 510, in response to receiving the first request and the type of the first sub-service that the first distributed computing environment is configured to support, the first kernel initiates a provisioning process of the specified resource in the deployment ring of the first distributed computing environment. The provisioning process includes obtaining an image based on the type of the first sub-service. The image is a software package template including specifications of resources used to run the software package, and the template is customized to use the specified resource based on the type of the first sub-service. Obtaining the image includes identifying an image from a plurality of images based on the type of the first sub-service. The image is configured with security best practices to reduce image vulnerabilities, and the security best practices are selected based on the type of the first sub-service. In some cases, the image is further configured with image specifications to optimize deployment workflow and performance, and the image specifications are selected based on the type of the first sub-service.
[0077] The provisioning process further includes deploying specified resources in the deployment ring, which includes parsing the image to identify the specified resources, requesting available resources from the deployment ring, parsing the available resources from the deployment ring, and determining whether all of the specified resources from the image are available in the deployment ring based on the parsing of the available resources. If all of the specified resources from the image are not available in the deployment ring, auto-scaling the available resources based on the image to provide additional resources to the deployment ring to satisfy the specified resources identified from the image. If all of the specified resources from the image are available in the deployment ring, reserving the specified resources.
[0078] The provisioning process further includes using the image to build one or more containers that encapsulate the specified resources. The one or more containers are wrapped into a first pod for running one or more programs to provide the first subservice. Once the one or more containers are built, the image is modified using a writable layer to include source code necessary to run the one or more programs. In certain cases, the image is modified to include source code, libraries, dependencies, tools, and other files necessary to run the one or more programs to provide the first subservice.
[0079] In step 515, the first sub-service is deployed using one or more replicas of the first pod that includes the specified provisioned resources and the modified image for the type of the first sub-service. The first sub-service is deployed in the first distributed computing environment.
[0080] In step 520, a health check is performed to evaluate the functionality of the first subservice and determine whether the first subservice is functioning as expected. If the source code and executable program are valid (meeting all or part of the requirements of the QMS), the source code and executable program do not contain open source vulnerabilities (as defined and determined by the software platform in accordance with the QMS), and / or the first subservice passes the health check, the first subservice is functioning as expected. If the first subservice fails the health check, the CICD system is notified regarding the service not functioning as expected. If the first subservice is functioning as expected, (i) the first subservice is published, (ii) security for the first subservice is initiated, and (iii) the CICD system is notified regarding the success of the first subservice functioning as expected.
[0081] Optionally, in step 525, if the first sub-service is functioning as expected, the first sub-service may be offered for use, and a user may begin execution of the first sub-service. Execution of the first sub-service may include (i) determining, by the kernel's master, a framework for executing the first sub-service based on the first sub-service or service type; (ii) offering, by the kernel's master, available resources in the placement ring to the framework for executing one or more programs to provide the first sub-service; receiving, by the kernel's master in response to the offer, information regarding tasks defined by the one or more programs; and sending, by the kernel's master, the tasks to a kernel's agent, which allocates the specified resources to an executor of the framework for executing the tasks.
[0082] 5, processes similar to those of steps 505-525 may be performed for second, third, fourth, etc. (n) sub-services of a service in parallel, substantially parallel, offset, or sequential order with one another and with the processes performed for the first sub-service. These processes may be performed to deploy all or a portion of the sub-services associated with a service deployed on a software platform across multiple distributed computing environments, in accordance with aspects of the present disclosure.
[0083] V. Further Considerations Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.
[0084] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude any equivalents of the features shown and described, or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.
[0085] The following description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of preferred exemplary embodiments will provide those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the appended claims.
[0086] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
Claims
1. 1. A computer-implemented method comprising: receiving, at a first kernel residing in a first distributed computing environment, a first request to initiate a deployment process for a first subservice or service on the first distributed computing environment, the first subservice or service having a type, and wherein in order for the first subservice or service to be deployed on the first distributed computing environment, the type of the first subservice or service must be one that the first distributed computing environment is configured to support; provisioning, by the first kernel, a specified resource within a placement ring of the first distributed computing environment in response to receiving the request and the type of the first subservice or service that the first distributed computing environment is configured to support, wherein the provisioning includes: obtaining an image based on the type of the first subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the first subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a first pod for executing one or more programs to provide the first sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the first distributed computing environment, the provisioning comprising: deploying, by the first kernel, the first subservice or service using one or more replicas of the first pod and the modified image; 11. A computer-implemented method comprising:
2. locating the resource, analyzing the image to identify the specified resource; requesting available resources from the placement ring; parsing the available resources from the placement ring; determining whether all of the specified resources from the image are available within the placement ring based on the parsing of the available resources; in response to all of the specified resources from the image not being available within the placement ring, auto-scaling the available resources based on the image to provide additional resources to the placement ring to satisfy the specified resources identified from the image; The computer-implemented method of claim 1 , comprising:
3. obtaining the image further includes identifying the image from a plurality of images based on the type of the first sub-service or service; the image is configured in accordance with security best practices to reduce vulnerabilities in the image; The computer-implemented method of claim 1 or 2, wherein the security best practice is selected based on the type of the first sub-service or service.
4. the image is further configured according to image specifications to optimize deployment workflow and performance; The computer-implemented method of claim 3 , wherein the image specification is selected based on the type of the first sub-service or service.
5. 5. The computer-implemented method of claim 1, wherein the type of the first sub-service or service is machine learning model software, software as a medical device, software that operates in conjunction with a physical medical device, or data capture and processing software.
6. and further comprising executing, by the kernel, the first sub-service or service, wherein the executing includes: determining, by a master of the kernel, a framework for executing the first subservice or service based on the type of the first subservice or service; offering, by the master of the kernel, available resources in the placement ring to the framework for executing the one or more programs to provide the first sub-service or service; receiving, by the master of the kernel in response to the offer, information regarding tasks defined by the one or more programs; sending, by the master of the kernel, the task to an agent of the kernel that assigns the specified resources to an execution unit of the framework for executing the task; The computer-implemented method of claim 5 , comprising:
7. Exposing, by the kernel, the first sub-service or service; initiating, by the kernel, security of the first sub-service or service; The computer-implemented method of claim 1 , further comprising:
8. performing, by the kernel, a health check to evaluate the functionality of the first subservice or service and determine whether the first subservice or service is functioning as expected; notifying a continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check, wherein the first request is received from the CICD system; notifying the continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check; When the first sub-service or service is functioning as expected, (i) publishing the first sub-service or service by the kernel; (ii) initiating security for the first sub-service or service by the kernel; and (iii) notifying the CICD system regarding the first sub-service or service's successful functioning as expected. The computer-implemented method of claim 1 , further comprising:
9. receiving, at a second kernel residing in a second distributed computing environment, a second request to initiate a deployment process for a second sub-service of the service on the second distributed computing environment, the second sub-service having a type, and wherein in order for the second sub-service or service to be deployed on the second distributed computing environment, the type of the second sub-service or service must be one that the second distributed computing environment is configured to support; provisioning, by the second kernel, a specified resource within a placement ring of the second distributed computing environment in response to receiving the request, wherein the provisioning includes: obtaining an image based on the type of the second subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the second subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a second pod for executing one or more programs to provide the second sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the second distributed computing environment, the provisioning including: deploying, by the second kernel, the second subservice or service using one or more replicas of the second pod and the modified image; The computer-implemented method of claim 1 , further comprising:
10. 1. A system comprising: one or more data processors; A non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to: receiving, at a first kernel residing in a first distributed computing environment, a first request to initiate a deployment process for a first subservice or service on the first distributed computing environment, the first subservice or service having a type, and wherein in order for the first subservice or service to be deployed on the first distributed computing environment, the type of the first subservice or service must be one that the first distributed computing environment is configured to support; provisioning, by the first kernel, a specified resource within a placement ring of the first distributed computing environment in response to receiving the request and the type of the first subservice or service that the first distributed computing environment is configured to support, wherein the provisioning includes: obtaining an image based on the type of the first subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the first subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a first pod for executing one or more programs to provide the first sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the first distributed computing environment, the provisioning comprising: deploying, by the first kernel, the first subservice or service using one or more replicas of the first pod and the modified image; a non-transitory computer-readable storage medium for performing operations, the non-transitory computer-readable storage medium including: A system comprising:
11. locating the resource, analyzing the image to identify the specified resource; requesting available resources from the placement ring; parsing the available resources from the placement ring; determining whether all of the specified resources from the image are available within the placement ring based on the parsing of the available resources; in response to all of the specified resources from the image not being available within the placement ring, auto-scaling the available resources based on the image to provide additional resources to the placement ring to satisfy the specified resources identified from the image; The system of claim 10, comprising:
12. obtaining the image further includes identifying the image from a plurality of images based on the type of the first sub-service or service; the image is configured in accordance with security best practices to reduce vulnerabilities in the image; The system of claim 10 or 11, wherein the security best practice is selected based on the type of the first sub-service or service.
13. the image is further configured according to image specifications to optimize deployment workflow and performance; The system of claim 12 , wherein the image specification is selected based on the type of the first sub-service or service.
14. 14. The system of claim 10, wherein the first sub-service or type of service is machine learning model software, software as a medical device, software operating in conjunction with a physical medical device, or data capture and processing software.
15. The operations further include executing, by the kernel, the first sub-service or service, wherein the executing: determining, by a master of the kernel, a framework for executing the first subservice or service based on the type of the first subservice or service; offering, by the master of the kernel, available resources in the placement ring to the framework for executing the one or more programs to provide the first sub-service or service; receiving, by the master of the kernel in response to the offer, information regarding tasks defined by the one or more programs; sending, by the master of the kernel, the task to an agent of the kernel that assigns the specified resources to an execution unit of the framework for executing the task; The system of claim 14 , comprising:
16. Exposing, by the kernel, the first sub-service or service; initiating, by the kernel, security of the first sub-service or service; The system of claim 10 further comprising:
17. The operation is performing, by the kernel, a health check to evaluate the functionality of the first subservice or service and determine whether the first subservice or service is functioning as expected; notifying a continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check, wherein the first request is received from the CICD system; notifying the continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check; When the first sub-service or service is functioning as expected, (i) publishing the first sub-service or service by the kernel; (ii) initiating security for the first sub-service or service by the kernel; and (iii) notifying the CICD system regarding the first sub-service or service's successful functioning as expected. The system of claim 10 further comprising:
18. The operation is receiving, at a second kernel residing in a second distributed computing environment, a second request to initiate a deployment process for a second sub-service of the service on the second distributed computing environment, the second sub-service having a type, and wherein in order for the second sub-service or service to be deployed on the second distributed computing environment, the type of the second sub-service or service must be one that the second distributed computing environment is configured to support; provisioning, by the second kernel, a specified resource within a placement ring of the second distributed computing environment in response to receiving the request, wherein the provisioning includes: obtaining an image based on the type of the second subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the second subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a second pod for executing one or more programs to provide the second sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the second distributed computing environment, the provisioning including: deploying, by the second kernel, the second subservice or service using one or more replicas of the second pod and the modified image; 18. The system of claim 10, further comprising:
19. 1. A computer program product for causing one or more data processors to perform operations, said operations comprising: receiving, at a first kernel residing in a first distributed computing environment, a first request to initiate a deployment process for a first subservice or service on the first distributed computing environment, the first subservice or service having a type, and wherein in order for the first subservice or service to be deployed on the first distributed computing environment, the type of the first subservice or service must be one that the first distributed computing environment is configured to support; provisioning, by the first kernel, a specified resource within a placement ring of the first distributed computing environment in response to receiving the request and the type of the first subservice or service that the first distributed computing environment is configured to support, wherein the provisioning includes: obtaining an image based on the type of the first subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the first subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a first pod for executing one or more programs to provide the first sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the first distributed computing environment, the provisioning comprising: deploying, by the first kernel, the first subservice or service using one or more replicas of the first pod and the modified image; a computer program comprising:
20. locating the resource, analyzing the image to identify the specified resource; requesting available resources from the placement ring; parsing the available resources from the placement ring; determining whether all of the specified resources from the image are available within the placement ring based on the parsing of the available resources; in response to all of the specified resources from the image not being available within the placement ring, auto-scaling the available resources based on the image to provide additional resources to the placement ring to satisfy the specified resources identified from the image; 20. The computer program of claim 19, comprising:
21. obtaining the image further includes identifying the image from a plurality of images based on the type of the first sub-service or service; the image is configured in accordance with security best practices to reduce vulnerabilities in the image; 21. The computer program product of claim 19 or 20, wherein the security best practice is selected based on the type of the first sub-service or service.
22. the image is further configured according to image specifications to optimize deployment workflow and performance; 22. The computer program product of claim 21, wherein the image specification is selected based on the type of the first sub-service or service.
23. 23. The computer program of any one of claims 19 to 22, wherein the first sub-service or type of service is machine learning model software, software as a medical device, software operating in conjunction with a physical medical device, or data capture and processing software.
24. The operations further include executing, by the kernel, the first sub-service or service, wherein the executing: determining, by a master of the kernel, a framework for executing the first subservice or service based on the type of the first subservice or service; offering, by the master of the kernel, available resources in the placement ring to the framework for executing the one or more programs to provide the first sub-service or service; receiving, by the master of the kernel in response to the offer, information regarding tasks defined by the one or more programs; sending, by the master of the kernel, the task to an agent of the kernel that assigns the specified resources to an execution unit of the framework for executing the task; 24. The computer program of claim 23, comprising:
25. Exposing, by the kernel, the first sub-service or service; initiating, by the kernel, security of the first sub-service or service; 25. The computer program of any one of claims 19 to 24, further comprising:
26. The operation is performing, by the kernel, a health check to evaluate the functionality of the first subservice or service and determine whether the first subservice or service is functioning as expected; notifying a continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check, wherein the first request is received from the CICD system; notifying the continuous integration and continuous deployment (CICD) system regarding the first subservice or service not functioning as expected if the first subservice or service fails the health check; When the first sub-service or service is functioning as expected, (i) publishing the first sub-service or service by the kernel; (ii) initiating security for the first sub-service or service by the kernel; and (iii) notifying the CICD system regarding the first sub-service or service's successful functioning as expected.
26. The computer program of any one of claims 19 to 25, further comprising:
27. The operation is receiving, at a second kernel residing in a second distributed computing environment, a second request to initiate a deployment process for a second sub-service of the service on the second distributed computing environment, the second sub-service having a type, and wherein in order for the second sub-service or service to be deployed on the second distributed computing environment, the type of the second sub-service or service must be one that the second distributed computing environment is configured to support; provisioning, by the second kernel, a specified resource within a placement ring of the second distributed computing environment in response to receiving the request, wherein the provisioning includes: obtaining an image based on the type of the second subservice or service, the image being a template of the software package including specifications of resources to be used to run the software package, the template being customized to use the specified resources based on the type of the second subservice or service; placing the designated resource within the placement ring; using the image to build one or more containers encapsulating the specified resources, the one or more containers being wrapped into a second pod for executing one or more programs to provide the second sub-service or service; modifying the image to include source code necessary to execute the one or more programs using a writable layer; and provisioning designated resources within a deployment ring of the second distributed computing environment, the provisioning including: deploying, by the second kernel, the second subservice or service using one or more replicas of the second pod and the modified image; 27. The computer program of any one of claims 19 to 26, further comprising:
28. A non-transitory machine-readable storage medium having recorded thereon a computer program according to any one of claims 19 to 27.
