Microservice deployment in a distributed computing environment

Abstract

This paper discloses a computer-implemented method for deploying and allocating resources for microservices in a distributed computing environment. The distributed computing environment includes a microservice deployment scheduler and one or more compute nodes. The microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. The method includes receiving microservice constraints describing microservice computation tasks through the microservice deployment scheduler. The method also includes receiving node-specific attributes from one or more compute nodes through the microservice deployment scheduler. The node-specific attributes describe the computational capacity and / or computational capabilities of the one or more compute nodes. The method further includes orchestrating the operations of one or more compute nodes through the microservice deployment scheduler by inputting the microservice constraints and node-specific attributes into the reinforcement learning-based dynamic workload orchestration module.

Description

Background Technology

[0001] This invention relates to the field of digital computer systems, and more specifically, to a method for managing the deployment of microservices in a distributed computing system.

[0002] In distributed computing, such as edge computing, the challenge lies in managing the deployment lifecycle of microservice workloads across a large number of compute nodes, such as edge nodes. In this context, compute nodes have certain attributes assigned by (edge) administrators, while microservices have certain associated metadata defined by developers. This metadata, in turn, defines a set of constraints about edge nodes and their attributes, which in turn determine which microservices can ultimately be deployed to the designated edge nodes. Summary of the Invention

[0003] In one aspect, the present invention provides a computer-implemented method for deploying and allocating resources for microservices in a distributed computing environment. The distributed computing environment includes a microservice deployment scheduler and one or more compute nodes. The microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. The method includes receiving microservice constraints describing microservice computation tasks through the microservice deployment scheduler. The method also includes receiving node-specific attributes from the one or more compute nodes through the microservice deployment scheduler. The node-specific attributes describe the computational capacity and / or computational capabilities of the one or more compute nodes. The method further includes orchestrating the operations of the one or more compute nodes through the microservice deployment scheduler by inputting the microservice constraints and node-specific attributes into the reinforcement learning-based dynamic workload orchestration module.

[0004] In another aspect, the present invention provides a computer program product comprising a computer-readable storage medium having computer-readable program code implemented therewith. The computer-readable program code is configured to implement embodiments of the computer-implemented method.

[0005] In another aspect, the present invention provides a computer system. The computer system includes a processor configured to control the computer system. The computer system also includes a memory storing machine-executable instructions and a microservice deployment scheduler. The microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. Execution of the instructions causes the processor to receive microservice constraints describing microservice computation tasks via the microservice deployment scheduler. Execution of the instructions further causes the processor to receive node-specific attributes from one or more compute nodes via the microservice deployment scheduler. The node-specific attributes describe the computing capacity and computing power of the one or more compute nodes. Execution of the instructions further causes the processor to orchestrate the operations of one or more compute nodes via the microservice deployment scheduler by implementing the microservice constraints and node-specific attributes into the reinforcement learning-based dynamic workload orchestration module. Attached Figure Description

[0006] In the following, embodiments of the invention will be explained in more detail by way of example only, with reference to the accompanying drawings, wherein: Figure 1 An example of a computing environment is shown.

[0007] Figure 2 A cloud computing environment according to an example of the present invention is described.

[0008] Figure 3 An abstract model layer according to an example of the present invention is depicted.

[0009] Figure 4 It shows Figure 1 Another view of the computing environment shown.

[0010] Figure 5 A flowchart illustrating the method of using a computing environment is shown. Detailed Implementation

[0011] For illustrative purposes, various embodiments of the invention will be described, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles, practical application, or improvements of the embodiments relative to technologies found on the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0012] Examples can provide a computer-implemented approach to the deployment and resource allocation of microservices for a distributed computing environment. As used herein, a microservice can refer to a collection of software applications that may have one or more of the following properties: they can be deployed independently of each other, they can be loosely coupled, they can operate collectively, and in some instances they can be organized around a specific function or computational task. For example, microservices can be combined to perform larger tasks and can be used in different combinations. This can, for example, have the advantage of enabling the rapid development and deployment of different functionalities within a distributed computing environment. The distributed computing environment includes a microservice deployment scheduler.

[0013] As used in this article, a microservice deployment scheduler encompasses software components capable of scheduling services in a microservice or distributed computing environment. One or more compute nodes can be nodes that provide microservices. For example, in some cases, compute nodes can have or provide microservices as containerized applications. For instance, various virtual machines can exist on a particular compute node, which can be used to provide various microservices on demand.

[0014] The microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. This module works by being trained using reinforcement learning.

[0015] The method includes receiving microservice constraints describing microservice computation tasks via a microservice deployment scheduler. Microservice constraints define the computational requirements for performing a specific task. The method also includes receiving node-specific attributes from one or more compute nodes via the microservice deployment scheduler. Node-specific attributes describe the computational capacity and / or computational capabilities of the one or more compute nodes. Microservice constraints define computational requirements, and node-specific attributes essentially define the computational resources available on different nodes. The method further includes orchestrating the operations of one or more compute nodes via the microservice deployment scheduler by inputting the microservice constraints and node-specific attributes into a reinforcement learning-based dynamic workload orchestration module.

[0016] Because it uses a reinforcement learning-based dynamic workload orchestration module, the computer-implemented method can have one or more of the following advantages: 1. Examples show that effective resource utilization can be achieved by considering (edge) computing node attributes and microservice constraints, minimizing waste and improving the efficiency of the entire system.

[0017] 2. The example demonstrates how scalability can be ensured by dynamically assigning microservices to appropriate (edge) compute nodes, thereby achieving load balancing and accommodating an increasing number of microservices.

[0018] 3. The system demonstrates flexibility and adaptability through a dynamic adaptation module, allowing it to respond to changes in edge node attributes or microservice requirements and maintain an optimized deployment over time.

[0019] 4. This solution improves the quality of service delivered to end users by deploying microservices on (edge) computing nodes that can meet their specific needs, thereby enhancing performance and user experience.

[0020] The example also offers the following advantage: the reinforcement learning-based dynamic workload orchestration module has been trained to efficiently or effectively orchestrate the operations of one or more compute nodes. This can be particularly advantageous because in complex or large distributed computing environments, it may be impossible to specify rules for every combination of how to distribute workloads across different compute nodes, manage microservice deployments, or manage other aspects of the distributed computing environment. The reinforcement learning-based dynamic workload orchestration module can then be continuously adapted and learn how to better manage the different compute nodes.

[0021] In another example, the microservice deployment scheduler also includes a rule-based dynamic workload orchestration module configured to orchestrate the operations of one or more compute nodes using a predefined set of rules associated with microservice constraints and node-specific attributes. The method also includes orchestrating the operations of one or more compute nodes through the microservice deployment scheduler by inputting microservice constraints and node-specific attributes into the rule-based dynamic workload orchestration module. If the predefined set of rules is not associated with microservice constraints and node-specific attributes, a reinforcement learning-based dynamic workload orchestration module is used to orchestrate the operations of one or more compute nodes.

[0022] In this example, a rule-based system is first used to determine how to orchestrate workloads across the compute nodes. If the rules determine that a particular situation is not covered by the rules, a reinforcement learning-based dynamic workload orchestration module takes over and then distributes the workloads across the different compute nodes. This has the following advantages: For very simple or very common cases, a set of rules can be established. However, even when the rules do not cover certain things, this ensures that the microservice deployment scheduler continues to operate effectively and efficiently. As more is used, the reinforcement learning-based dynamic workload orchestration module can likely become even better, and it can adapt to specific distributed computing environments. This can provide a particularly efficient or effective distribution for allocating workloads across compute nodes.

[0023] In another example, a predefined rule set implements a multi-objective problem statement. A multi-objective problem statement defines an optimization problem involving two or more objective functions to be optimized simultaneously. In this example, the predefined rule set can select or distribute the load across the one or more computing nodes to optimize multiple objective functions or objectives. This can be advantageous because it provides a more flexible or efficient way to distribute computational load across different computing nodes.

[0024] In another example, the method also includes training a reinforcement learning-based dynamic workload orchestration module using a reward function during operation. While the reinforcement learning system is running, the reward function measures the system's operation, prediction, or control. By maximizing the value of the reward function, the reinforcement learning-based dynamic workload orchestration module is trained to consistently perform better. The reward function includes any of the following: resource utilization of compute nodes, response time to microservice constraints, cost efficiency of compute nodes, energy consumption of compute nodes, and combinations thereof. This example may be particularly useful because multiple of these values ​​can be used to construct the reward function. Therefore, a reinforcement learning-based dynamic workload orchestration module can provide a large number or variety of different metrics for optimizing the success or effectiveness of one or more compute nodes.

[0025] In another example, the distributed computing environment is configured to receive microservice applications from a microservice database for execution by one or more compute nodes. For instance, the microservice database can be distributed from a central location to different distributed computing environments. The microservice database can provide a centralized way to update or deliver microservices to the distributed computing environment.

[0026] In another example, the microservice deployment scheduler runs on a dedicated compute node. This dedicated compute node acts as a gateway between the microservice database and one or more compute nodes. In this example, one or more compute nodes are isolated from the microservice database via the microservice deployment scheduler. Therefore, the microservice deployment scheduler can provide security and a way to isolate one or more compute nodes. This can also, for example, provide a method for defining a distributed computing environment as an edge computing system, where each compute node is an edge computing node.

[0027] In edge computing, computing services are located near where they are used, rather than at a central server or central location.

[0028] In another example, the execution of the microservice deployment scheduler is distributed across one or more compute nodes. In this example, the functionality of the microservice deployment scheduler is distributed by positioning its compute resources across one or more compute nodes. This can, for example, provide a more cost-effective and flexible way to set up a distributed computing environment. In this example, no dedicated compute nodes are required to implement the microservice deployment scheduler.

[0029] In another example, the distributed computing environment includes a trained machine learning module configured to output node-specific attributes in response to receiving current-node-specific machine state data as input. The method includes collecting current-node-specific machine state data from one or more computing nodes and receiving node-specific attributes in response to inputting the current-node-specific machine state data into the trained machine learning module.

[0030] In this example, the trained machine learning module is used to acquire data describing a specific node (called current node-specific machine state data) and then outputs node-specific attributes. This can provide various benefits. The trained machine learning module can first be used to classify the state of a specific compute node. The classification of a compute node is its node-specific attribute. Another advantage of doing this is that node-specific attributes, rather than the complete current node-specific machine state data, can be transmitted to the microservice deployment scheduler. This enables efficient operation of the microservice deployment scheduler, where the amount of data between different nodes in a distributed computing environment is reduced. Instead of transmitting the complete current node-specific machine state data, only the node-specific attributes are transmitted.

[0031] In another example, the trained machine learning module includes a classifier neural network. This can be done using various types of neural networks that classify the output. For example, a neural network with multiple connected layers can be used. For instance, a fully connected neural network with at least three hidden layers can be used. In other examples, standard neural network architectures such as convolutional neural networks can also be used.

[0032] A classifier neural network can be trained by collecting current-node-specific machine state data from different computing nodes and then manually assigning node-specific attributes to the training data. Deep learning-type training routines can then be used. During training, loss functions such as binary cross-entropy loss, hinge loss, mean squared error loss, or L2 loss can be used.

[0033] In another example, the current node-specific machine state data includes any of the following: current resource utilization data, current workload data, current performance metric data, current processing power, current memory capacity of the specific compute node, current storage capacity of the node, current network bandwidth, current microservice resource requirements, current microservice response time target, current microservice execution dependencies, current microservice communication mode, currently installed microservices, current response time, current throughput, current energy consumption, current processing cost, and current storage cost. The term "current" used above is intended as a label to indicate data acquired within a predetermined time period or recently acquired data.

[0034] In another example, a trained machine learning module is implemented on each of one or more compute nodes. The method also includes receiving node-specific attributes in response to inputting node-specific machine state data to the trained machine learning module of the respective compute node. This example can be advantageous because the individual compute nodes will only transmit node-specific attributes, rather than the complete node-specific machine state data.

[0035] In another example, the trained machine learning module is also configured to receive historical machine state data as input when current node-specific machine state data is fed into the trained machine learning module. This can be advantageous because it provides a way to incorporate historical data, typical of a particular distributed computing environment, into the way a microservice deployment scheduler distributes jobs. The method also includes feeding historical machine state data into the trained machine learning module while current node-specific machine state data is fed into the trained machine learning module to determine node-specific attributes.

[0036] In another example, historical machine state data includes any of the following: historical resource utilization data, historical workload data, historical performance metric data, historical processing capacity, historical memory capacity, historical storage capacity, historical network bandwidth, historical microservice resource requirements, historical microservice response time targets, historical microservice execution dependencies, historical microservice communication patterns, historically installed microservices, historical response times, historical throughput, historical energy consumption, historical processing costs, and historical storage costs. The term "historical" used above is intended as a label to indicate data acquired before the predetermined time period described with "current data" or data that is not recently acquired. It can be archived data.

[0037] In another embodiment, the distributed computing environment is a distributed edge computing environment. The compute nodes are edge nodes. A distributed edge computing environment is a distributed computing environment where processing is distributed closer to the point where data is used or created, rather than located at a central server. This can, for example, provide greater robustness under high network traffic, or be used to maintain highly responsive and flexible microservice deployment systems.

[0038] Various aspects of this disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). Regarding any flowchart, depending on the technology involved, operations may be performed in a different order than that shown in a given flowchart. For example, again according to the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0039] Computer Program Product Embodiment (“CPP Embodiment” or “CPP”) is a term used in this disclosure to describe any collection of one or more storage media (also referred to as “media”) collectively included in a collection of one or more storage devices, the collection of one or more storage devices collectively including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device capable of holding and storing instructions used by a computer processor. Without limitation, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punch cards or pits / platforms formed in the main surface of the disk), or any suitable combination of the foregoing. Computer-readable storage media, as used in this disclosure, should not be construed as storing transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As those skilled in the art will understand, data is typically moved at certain incidental points in time during the normal operation of the storage device, such as during access, defragmentation, or garbage collection; however, this does not render the storage device transient, as the data is not transient when it is stored.

[0040] The computing environment 100 includes examples of environments for executing at least some of the computer code involved in performing inventive methods, such as the microservice deployment scheduler 400. In addition to block 400, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end-user equipment (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor set 110 (including processing circuitry 120 and a cache 121), a communication infrastructure 111, volatile memory 112, persistent storage 113 (including an operating system 122 and block 200, as described above), a peripheral device set 114 (including a user interface (UI) device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0041] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases such as remote database 130. As is well known in the field of computer technology, and depending on that technology, the execution of computer-implemented methods can be distributed among multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the detailed discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 can reside in the cloud, even... Figure 1 It is not shown that it is in the cloud. On the other hand, unless explicitly instructed otherwise, computer 101 is not required to be in the cloud.

[0042] Processor set 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple cooperating integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by the threads or cores running on processor set 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache in the processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to work with qubits and perform quantum computing.

[0043] Computer-readable program instructions are typically loaded onto computer 101 to cause the processor set 110 of computer 101 to perform a series of operational steps to implement a computer-implemented method, such that the instructions, when executed, instantiate the method specified in the flowcharts and / or narrative descriptions of the computer-implemented method included in this document (collectively, the “inventive method”). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the inventive method.

[0044] In computing environment 100, at least some of the instructions for implementing the method using microservice deployment scheduler 400 can be stored in persistent storage 113. For example, the instructions can be used to control the computing environment to manage the deployment and resource allocation of microservices in a distributed computing environment. The distributed computing environment includes a microservice deployment scheduler and one or more compute nodes. The microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. The method includes: receiving microservice constraints describing microservice computing tasks through the microservice deployment scheduler; receiving node-specific attributes from one or more compute nodes through the microservice deployment scheduler, wherein the node-specific attributes describe the computing capacity and / or computing power of one or more compute nodes; and orchestrating the operations of one or more compute nodes by inputting the microservice constraints and node-specific attributes into the reinforcement learning-based dynamic workload orchestration module through the microservice deployment scheduler.

[0045] Communication structure 111 is a signal transmission path that allows various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths forming buses, bridges, physical input / output ports, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0046] Volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamically typed random access memory (RAM) or statically typed RAM. Typically, volatile memory 112 is characterized by random access, but this is not required unless explicitly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally relative to computer 101.

[0047] Persistent storage 113 is any form of non-volatile storage for a computer, now known or to be developed in the future. The non-volatility of this storage means that the stored data is retained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage 113. Persistent storage 113 may be read-only memory (ROM), but typically at least a portion of persistent storage allows for data writing, data deletion, and data rewriting. Some common forms of persistent storage include disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or operating systems employing an open-source portable operating system interface type with a kernel. The code included in block 200 generally includes at least some of the computer code involved in performing the inventive methods described herein.

[0048] Peripheral device set 114 includes a collection of peripheral devices for computer 101. Data communication connections between peripheral devices and other components of computer 101 can be implemented in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections made by cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., secure digital (SD) cards), connections made via local area communication networks, and even connections made via wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as displays, speakers, microphones, wearable devices (such as glasses and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 124 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. Storage 124 can be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 requires a large amount of storage (e.g., where computer 101 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in IoT applications. For example, one sensor could be a thermometer, while another could be a motion detector.

[0049] Network module 115 is a collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers via WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi transceiver, software for packetizing and / or depacketizing data for transmission over the communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the methods of the invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in network module 115.

[0050] WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any technology known now or developed in the future for transmitting computer data. In some embodiments, WAN 102 may be replaced by and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0051] End User Equipment (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and can take any of the forms discussed above in conjunction with computer 101. EUD 103 typically receives helpful and useful data from the operation of computer 101. For example, assuming computer 101 is designed to provide recommendations to the end user, these recommendations are typically transmitted from network module 115 of computer 101 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present recommendations to the end user. In some embodiments, EUD 103 can be a client device, such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0052] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 can be controlled and used by the same entity operating computer 101. Remote server 104 represents a machine that collects and stores helpful and useful data used by other computers, such as computer 101. For example, if computer 101 is designed and programmed to provide recommendations based on historical data, that historical data can be provided to computer 101 from a remote database 130 of remote server 104.

[0053] Public cloud 105 is any computer system that can be used by multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities (especially data storage (cloud storage) and computing power) without direct active management by users. Cloud computing typically leverages resource sharing to achieve scalability consistency and economy. Direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers constituting host physical set 142, which is the entirety of physical computers in and / or available to the public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCEs are instantiated. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that allow public cloud 105 to communicate via WAN 102.

[0054] Now, we will provide some further explanation of Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of the programs running within them, these isolated user-space instances typically appear as actual computers. Computer programs running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this is a characteristic known as containerization.

[0055] Private cloud 106 is similar to public cloud 105, except that computing resources are available only to a single enterprise. While private cloud 106 is depicted as communicating with WAN 102, in other embodiments, private cloud may be completely disconnected from the Internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardization or proprietary technology that enables orchestration, management, and / or data / application portability across the multiple component clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0056] Figure 2 It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the doctrines described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0057] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0058] The characteristics are as follows: On-demand self-service: Cloud consumers can automatically and unilaterally provision computing power, such as server time and network storage, without requiring human interaction with the service provider. Broadband network access: Capabilities are available via the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or fat client platforms (e.g., mobile phones, laptops, and PDAs). Resource sharing: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated based on demand. Location independence exists, meaning that consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0059] Rapid Elasticity: Capacity can be provisioned quickly and elastically, sometimes automatically, to scale up and down rapidly. To consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time. Measurable Service: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at an abstraction level appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the service provider and the consumer. Service Models: Software as a Service (SaaS): The capability offered to consumers is the use of a provider's applications running on cloud infrastructure. Applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, and possibly even individual application capabilities, except for limited user-specific application configuration settings.

[0060] Platform as a Service (PaaS): Provides consumers with the ability to deploy consumer-created or acquired applications, built using provider-supported programming languages ​​and tools, onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage; instead, they have control over the deployed applications and, possibly, the configuration of the application hosting environment. Infrastructure as a Service (IaaS): Provides consumers with the ability to provision processing, storage, networking, and other basic computing resources, where consumers can deploy and run arbitrary software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure; instead, they have limited control over the operating system, storage, deployed applications, and, possibly, selected network components (such as host firewalls).

[0061] Deployment models are as follows: Private Cloud: The cloud infrastructure is operated solely by an organization. It can be managed by the organization or a third party and can exist internally or externally. Community Cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist internally or externally. Public Cloud: The cloud infrastructure is available to the public or large industry groups and is owned by an organization that sells cloud services. Hybrid Cloud: The cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds). Cloud computing environments are service-oriented, emphasizing statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0062] Now for reference Figure 2 The illustration shows a cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 that communicate with local computing devices used by cloud consumers, such as, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private, community, public, or hybrid clouds as described above, or a combination thereof. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service, eliminating the need for cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 2 The types of computing devices 54A-N shown are intended to be illustrative only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device across any type of network and / or network-addressable connectivity (e.g., using a web browser).

[0063] Now for reference Figure 3 This demonstrates a cloud computing environment of 50 ( Figure 1 This provides a set of functional abstraction layers. It should be understood beforehand that... Figure 3 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As shown, the following layers and corresponding functions are provided: The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) architecture server 62; a server 63; a blade server 64; a storage device 65; and a network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0064] The virtualization layer 70 provides an abstraction layer from which examples of the following virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0065] In one example, management layer 80 can provide the functionality described below. Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks in the cloud computing environment. Metering and Pricing 82 provides cost tracking when resources are used in the cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to meet required service levels. Service Level Agreement (SLA) Planning and Implementation 85 provides pre-planning and procurement of cloud resources for anticipated future needs to comply with the SLA.

[0066] Workload 90 provides examples of functionalities that can be performed using a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics and processing 94; transaction processing 95; and others. Figure 1 The context describes microservice deployment and scheduling 96.

[0067] Figure 4 A further view of computing environment 100 is shown. A microservice deployment scheduler 400 is implemented on computer 101. As mentioned above, microservices are executable programs that can be deployed independently of each other, can be loosely coupled, can operate together, and in some instances, can be organized around specific capabilities or computing tasks.

[0068] Computing environment 100 is shown as comprising a distributed computing environment 402, which includes a computer 101 and multiple computing nodes 404. Network module 115 is shown as forming a network connection with each computing node 404 and a microservice database 406. Microservice database 406 can provide microservices 420 to computing nodes 404.

[0069] Compute node 404 is shown as an implementation containing various microservices 420. Compute node 404 is capable of performing computational tasks using the microservices 420 in a distributed and potentially collaborative manner. Each node may have different microservices. As different microservices 420 are distributed to them and as machine states change over time, the computational power of each node may change over time.

[0070] Compute node 404 is shown as containing node-specific machine state data 424 describing the computation and computational resources available for a particular compute node 404. Therefore, the node-specific machine state data 424 describes the current computational state or capacity of compute node 404.

[0071] However, node-specific machine state data 424 is raw data and may not concisely describe the current computing power of compute node 404. The compute node is shown as having a trained machine learning module 422. The trained machine learning module takes node-specific machine state data 424 as input and outputs node-specific attribute 426 as a response. In some examples, node-specific attribute 426 can be considered a classification of the computing power of compute node 404. This can include the current workload and which microservices are available to perform tasks at compute node 404. For example, the trained machine learning module 422 can be a classifier neural network. It receives node-specific machine state data 424 and outputs node-specific attribute 426 as a classification. The node-specific attribute 426 can then be transmitted via network module 115 of computer 101 and stored in memory 113. Transmitting node-specific attribute 426 may require fewer network resources compared to transmitting the entire node-specific machine state data 424.

[0072] The computer's memory 113 is shown as containing a microservice deployment scheduler 410. The microservice deployment scheduler 140 uses node-specific machine state data 424 to perform orchestration of compute nodes 404. The microservice deployment scheduler attempts to reconcile microservice constraints 416 (the requested capacity to perform compute tasks) with the available compute resources of the compute nodes described by the microservice deployment scheduler 410. The microservice deployment scheduler 410 optionally includes a rule-based dynamic rule-based optimization module 414 and also includes a reinforcement learning-based dynamic workload orchestration module 412. If the rule-based dynamic rule-based optimization module 414 is present, it is typically attempted to perform orchestration first, and if the rules used by the rule-based dynamic rule-based optimization module 414 are insufficient to match the microservice constraints 416 with the available compute resources (as specified in node-specific attribute 426), the reinforcement learning-based dynamic workload orchestration module 412 is used. During the use of the reinforcement learning-based dynamic workload orchestration module 412, the reward function 418 can be used to train the module 412 using reinforcement learning.

[0073] Network module 115 is also shown connected to microservice database 406. Computer 101 acts as a gateway between microservice database 406 and compute node 404. In this configuration, computer 101 serves as a means of isolating compute node 404 from the larger network. For example, this can be advantageous in reducing the vulnerability of compute node 404 to attacks and in discovering the topology of the edge computing environment.

[0074] Figure 5 A flowchart is shown, which illustrates the operation. Figure 4The method of the computing environment 100 shown. In step 500, current node-specific machine state data 424 is collected from one or more computing nodes 404. This can be as follows: Figure 4 The process is performed as shown, where the data is collected locally. In other examples, the data may be forwarded to computer 101. In step 502, node-specific attributes 426 are received in response to inputting current node-specific machine state data 424 into the trained machine learning module 422. In step 404, computer 101 receives microservice constraints 416. These may be provided to the microservice deployment scheduler 410. In step 506, node-specific attributes 426 are received. For example, these may be received directly from compute node 404. In step 508, orchestration of one or more compute nodes by the microservice deployment scheduler 400 is performed by inputting the microservice constraints 416 and node-specific attributes 426 into the rule-based dynamic rule-based optimization module 414.

[0075] In some examples, step 508 is optional. The rules in the rule-based dynamic optimization module 414 may succeed or fail. For example, if a rule does not exist for a particular case, that particular module 414 may fail. Box 510 is the decision box. The question is: Did the rule succeed? If the answer is "yes," then orchestration has been performed, and the method proceeds to box 516, which is the final step. If the answer is "no," then the reinforcement learning-based dynamic workload orchestration module 412 is used. In this context, rule success means that the rule is applicable to solving constraints and node-specific machine state data. If you have a rule applicable to a particular case, then orchestration is performed according to the rule. If the rule is not applicable, the reinforcement learning-based dynamic orchestration module takes over.

[0076] In step 512, the microservice deployment scheduler 410 orchestrates the operations of one or more compute nodes 404 by inputting microservice constraints 416 and node-specific attributes 426 into the reinforcement learning-based dynamic workload orchestration module 412. In an optional step 514, the reinforcement learning-based dynamic workload orchestration module 412 is trained using a reward function 418 during the operation.

[0077] For example, a reinforcement learning-based dynamic workload orchestration module 412 can be implemented using a deep Q-neural network (DQNN) for handling RL tasks. In this case, the RL tasks could cover the deployment and resource allocation of microservices in a distributed computing environment. The classic Bellman equation for the Q-value can be enhanced by a weight vector w (representing the weights and parameters of the DQNN) to measure the overall reward for actions taken within the system, i.e. , where alpha[0,1] is a discount factor, as is commonly used in Q-learning, and beta[0,1] is an additional factor that can be used to converge future reward estimates to the measured values, and optionally can be learned for better performance and stability. Using the Q-objective function described above, we continue by defining a mean squared error (MSE) metric for the loss function of the DQNN, i.e. Next, the DQNN is trained for a specified number of epochs or until convergence based on available observations (some collected datasets), ultimately producing policy π for optimal deployment and resource allocation of microservices in a distributed computing environment.

[0078] Incorporating reinforcement learning (RL) into microservice orchestration using deep Q-networks (DQNs) can involve defining appropriate states, actions, and rewards to capture the dynamics of a distributed microsystem.

[0079] State is the current observation of the system, such as the current workload at the client, potential resource usage, and the number of deployed microservices. This can encompass the number or attributes of the system that can be monitored. For example, state can include one or more of the following: • Resource utilization: CPU, memory, disk I / O, and network I / O for each microservice instance.

[0080] • Service performance metrics: response time, throughput, and error rate.

[0081] • System load: The overall load on the system, which can include factors such as requests per second.

[0082] • Microservice health status: Indicators such as up / down status, number of restarts, etc.

[0083] • Dependency status: The health status and performance of dependent services or databases.

[0084] Actions are policies / rules that determine when to schedule which microservices in this system. This can include adaptable content and can involve anything from self-removing services and redeploying services to orchestrating microservice deployments, splitting, reallocating, etc. For example, actions can include one or more of the following: • Scaling action: Increase or decrease the number of microservice instances.

[0085] • Resource allocation: Increase or decrease the resources (CPU, memory) allocated to microservices.

[0086] • Load balancing adjustment: Modify load balancing policies or rules.

[0087] • Service restart / recovery actions: when the service is shut down or has poor performance.

[0088] • Configuration changes: Adjust the configuration parameters of microservices or orchestration platforms.

[0089] The reward is a measurable quantity. In this case, the reward can be closely related to the running service, such as the inference quality of the ML task, or something very simple, such as "Do I still have 2GB of RAM left?" For example, the reward can include one or more of the following: • Performance improvements: Positive rewards are given for actions that lead to improved response time or throughput.

[0090] • Cost efficiency: Rewards are given for reducing resource usage without affecting performance.

[0091] • Stability: Positive rewards are given for maintaining or improving the stability and health of the system.

[0092] • SLA compliance: Reward actions that help meet or exceed the defined service level agreement.

[0093] • Penalty: Negative rewards are given for actions that lead to decreased performance, increased costs, or system instability.

[0094] In the context of DQN, these definitions can be translated into numerical values ​​that a neural network can process. For example, resource utilization can be expressed as a percentage, and service performance can be measured in milliseconds or transactions per second. In some examples, this can be achieved by encoding these metrics into state representations that DQN can understand and learn. During reinforcement learning, additional node-specific machine state data 424 and / or node-specific attributes 426 can be collected to compute the value of the reward function during training.

[0095] Various examples can be described by one or more of the following features in the following numbered clauses: Clause 1. A computer-implemented method for deploying and allocating resources for microservices in a distributed computing environment, wherein the distributed computing environment includes a microservice deployment scheduler and one or more compute nodes, wherein the microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. The method includes: The microservice deployment scheduler receives microservice constraints describing microservice computation tasks. The microservice deployment scheduler receives node-specific attributes from the one or more compute nodes, wherein the node-specific attributes describe the computing power and / or computing capacity of the one or more compute nodes; and By inputting the microservice constraints and the node-specific attributes into the reinforcement learning-based dynamic workload orchestration module, the operations of one or more computing nodes are orchestrated by the microservice deployment scheduler.

[0096] Clause 2. The computer-implemented method of Clause 1, wherein the microservice deployment scheduler further includes a rule-based dynamic workload orchestration module configured to orchestrate the operations of the one or more compute nodes using a predetermined set of rules associated with the microservice constraints and the node-specific attributes, wherein the method further includes orchestrating the one or more compute nodes by the microservice deployment scheduler through the rule-based dynamic workload orchestration module by inputting the microservice constraints and the node-specific attributes, wherein if the predetermined set of rules is not associated with the microservice constraints and the node-specific attributes, the operation of orchestrating the one or more compute nodes using the reinforcement learning-based dynamic workload orchestration module is performed.

[0097] Clause 3. The computer-implemented method according to Clause 2, wherein the predetermined set of rules implements a multi-objective problem statement.

[0098] Clause 4. A computer-implemented method according to Clause 1, 2 or 3, wherein the method further comprises training the reinforcement learning-based dynamic workload orchestration module during operation using a reward function, wherein the reward function includes any one of the following: resource utilization of the compute node, response time to the microservice constraints, cost efficiency of the compute node, energy consumption of the compute node and combinations thereof.

[0099] Clause 5. A computer-implemented method according to any of the preceding clauses, wherein the distributed computing environment is configured to receive microservice applications from a microservice database for execution by the one or more computing nodes.

[0100] Clause 6. The computer-implemented method according to Clause 5, wherein the microservice deployment scheduler is executed on a dedicated computing node, wherein the dedicated computing node serves as a gateway between the microservice database and the one or more computing nodes.

[0101] Clause 7. The computer-implemented method according to any one of Clauses 1 to 5, wherein the execution of the microservice deployment scheduler is distributed across the one or more computing nodes.

[0102] Clause 8. A computer-implemented method according to any of the preceding clauses, wherein the distributed computing environment includes a trained machine learning module configured to output a node-specific attribute in response to receiving current node-specific machine state data as input, wherein the method further includes: Collect specific machine state data of the current node from the one or more computing nodes; and In response to inputting the current node-specific machine state data into the trained machine learning module, the node-specific attributes are received.

[0103] Clause 9. The computer-implemented method according to Clause 8, wherein the trained machine learning module includes a classifier neural network.

[0104] Clause 10. The computer-implemented method according to Clause 8 or 9, wherein the current node-specific machine state data includes any one of the following: current resource utilization data, current workload data, current performance metric data, current processing power, current memory capacity, current storage capacity, current network bandwidth, current microservice resource requirements, current microservice response time target, current microservice execution dependencies, current microservice communication mode, currently installed microservices, current response time, throughput, current energy consumption, current processing cost, current storage cost, and combinations thereof.

[0105] Clause 11. A computer-implemented method according to Clause 8, 9 or 10, wherein the trained machine learning module is implemented on each of the one or more computing nodes, wherein the method further includes receiving node-specific attributes in response to inputting node-specific machine state data into the trained machine learning module of the respective computing node.

[0106] Clause 12. A computer-implemented method according to any one of Clauses 8 to 10, wherein the trained machine learning module is further configured to receive historical machine state data as input when the current node-specific machine state data is input to the trained machine learning module, wherein the method further includes inputting the historical machine state data into the trained machine learning module when the current node-specific machine state data is input to the trained machine learning module to determine the node-specific attribute.

[0107] Clause 13. The computer-implemented method according to Clause 12, wherein the historical machine state data includes any one of the following: historical resource utilization data, historical workload data, historical performance metric data, historical processing capacity, historical memory capacity, historical storage capacity, historical network bandwidth, historical microservice resource requirements, historical microservice response time targets, historical microservice execution dependencies, historical microservice communication patterns, historically installed microservices, historical response times, historical throughput, historical energy consumption, historical processing costs, historical storage costs, and combinations thereof.

[0108] Clause 14. The computer-implemented method according to any of the preceding clauses, wherein the distributed computing environment is a distributed edge computing environment, and wherein the computing node is an edge node.

[0109] Clause 15. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied thereon, the computer-readable program code being configured to implement the method described in any one of Clauses 1 to 14.

[0110] Clause 16. A computer system comprising: A processor, configured to control the computer system; and A memory storing machine-executable instructions and a microservice deployment scheduler, wherein the microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module, wherein executing the instructions causes the processor to: The microservice deployment scheduler receives microservice constraints describing microservice computation tasks. The microservice deployment scheduler receives node-specific attributes from the one or more compute nodes, wherein the node-specific attributes describe the computing power and computational capabilities of the one or more compute nodes; and By inputting the microservice constraints and the node-specific attributes into the reinforcement learning-based dynamic workload orchestration module, the operations of one or more computing nodes are orchestrated by the microservice deployment scheduler.

[0111] Clause 17. The computer system of Clause 16, wherein the microservice deployment scheduler further includes a rule-based dynamic workload orchestration module configured to orchestrate the operations of the one or more compute nodes using a predetermined set of rules associated with the microservice constraints and the node-specific attributes, wherein executing the instructions causes the processor to orchestrate the operations of the one or more compute nodes by inputting the microservice constraints and the node-specific attributes into the rule-based dynamic workload orchestration module, wherein if the predetermined set of rules is not associated with the microservice constraints and the node-specific attributes, the processor performs the orchestration of the operations of the one or more compute nodes using the reinforcement learning-based dynamic workload orchestration module.

[0112] Clause 18. The computer system pursuant to Clause 17, wherein the predetermined set of rules implements a multi-objective problem statement.

[0113] Clause 19. A computer system according to Clauses 16, 17 or 18, wherein executing the instructions further causes the processor to train the reinforcement learning-based dynamic workload orchestration module using a reward function during operation, wherein the reward function includes any one of the following: resource utilization of the compute node, response time to the microservice constraints, cost efficiency of the compute node, energy consumption of the compute node, and combinations thereof.

[0114] Clause 20. A computer system pursuant to any one of Clauses 16 to 19, wherein the distributed computing environment is configured to receive microservice applications from a microservice database for execution by the one or more compute nodes, wherein the microservice deployment scheduler is executed on a dedicated compute node, wherein the dedicated compute node serves as a gateway between the microservice database and the one or more compute nodes.

Claims

1. A computer-implemented method for deploying and allocating resources for microservices in a distributed computing environment, wherein the distributed computing environment includes a microservice deployment scheduler and one or more computing nodes, wherein the microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module. The method includes: The microservice deployment scheduler receives microservice constraints describing microservice computation tasks. The microservice deployment scheduler receives node-specific attributes from the one or more compute nodes, wherein the node-specific attributes describe the computing capacity and / or computing power of the one or more compute nodes; as well as By inputting the microservice constraints and the node-specific attributes into the reinforcement learning-based dynamic workload orchestration module, the operations of one or more computing nodes are orchestrated by the microservice deployment scheduler.

2. The computer-implemented method of claim 1, wherein the microservice deployment scheduler further comprises a rule-based dynamic workload orchestration module configured to orchestrate the operations of the one or more compute nodes using a predetermined set of rules associated with the microservice constraints and the node-specific attributes, wherein the method further comprises orchestrating the one or more compute nodes by inputting the microservice constraints and the node-specific attributes into the rule-based dynamic workload orchestration module via the microservice deployment scheduler, wherein if the predetermined set of rules is not associated with the microservice constraints and the node-specific attributes, the operation of orchestrating the one or more compute nodes using the reinforcement learning-based dynamic workload orchestration module is performed.

3. The computer-implemented method according to claim 2, wherein the predetermined rule set implements a multi-objective problem statement.

4. The computer-implemented method according to any one of the preceding claims, wherein the method further comprises training the reinforcement learning-based dynamic workload orchestration module using a reward function during operation, wherein the reward function includes any one of the following: resource utilization of the compute node, response time to the microservice constraints, cost efficiency of the compute node, energy consumption of the compute node, and combinations thereof.

5. The computer-implemented method according to any one of the preceding claims, wherein the distributed computing environment is configured to receive microservice applications from a microservice database for execution by the one or more computing nodes.

6. The computer-implemented method of claim 5, wherein the microservice deployment scheduler is executed on a dedicated computing node, wherein the dedicated computing node serves as a gateway between the microservice database and the one or more computing nodes.

7. The computer-implemented method according to any one of the preceding claims, wherein the execution of the microservice deployment scheduler is distributed across the one or more computing nodes.

8. The computer-implemented method according to any one of the preceding claims, wherein the distributed computing environment includes a trained machine learning module configured to output a node-specific attribute in response to receiving current node-specific machine state data as input, wherein the method further comprises: Collect specific machine status data of the current node from the one or more computing nodes; as well as In response to inputting the current node-specific machine state data into the trained machine learning module, the node-specific attributes are received.

9. The computer-implemented method of claim 8, wherein the trained machine learning module comprises a classifier neural network.

10. The computer-implemented method according to claim 8, wherein the current node-specific machine state data includes any one of the following: current resource utilization data, current workload data, current performance metric data, current processing power, current memory capacity, current storage capacity, current network bandwidth, current microservice resource requirements, current microservice response time target, current microservice execution dependencies, current microservice communication mode, currently installed microservices, current response time, throughput, current energy consumption, current processing cost, current storage cost, and combinations thereof.

11. The computer-implemented method of claim 8, wherein the trained machine learning module is implemented on each of the one or more computing nodes, wherein the method further comprises receiving the node-specific attribute in response to inputting the node-specific machine state data to the trained machine learning module of the respective computing node.

12. The computer-implemented method of claim 8, wherein the trained machine learning module is further configured to receive historical machine state data as input when the current node-specific machine state data is input to the trained machine learning module, wherein the method further includes inputting the historical machine state data to the trained machine learning module when the current node-specific machine state data is input to the trained machine learning module to determine the node-specific attribute.

13. The computer-implemented method of claim 12, wherein the historical machine state data includes any one of the following: historical resource utilization data, historical workload data, historical performance metric data, historical processing capacity, historical memory capacity, historical storage capacity, historical network bandwidth, historical microservice resource requirements, historical microservice response time targets, historical microservice execution dependencies, historical microservice communication patterns, historically installed microservices, historical response time, historical throughput, historical energy consumption, historical processing costs, historical storage costs, and combinations thereof.

14. The computer-implemented method according to any one of the preceding claims, wherein the distributed computing environment is a distributed edge computing environment, and wherein the computing node is an edge node.

15. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied thereon, the computer-readable program code being configured to implement the method according to any one of the preceding claims.

16. A computer system comprising: A processor configured to control the computer system; as well as The memory stores machine-executable instructions and a microservice deployment scheduler, wherein the microservice deployment scheduler includes a reinforcement learning-based dynamic workload orchestration module, wherein executing the instructions causes the processor to: The microservice deployment scheduler receives microservice constraints describing microservice computation tasks. The microservice deployment scheduler receives node-specific attributes from the one or more compute nodes, wherein the node-specific attributes describe the computing capacity and computing power of the one or more compute nodes; as well as By inputting the microservice constraints and the node-specific attributes into the reinforcement learning-based dynamic workload orchestration module, the operations of one or more computing nodes are orchestrated by the microservice deployment scheduler.

17. The computer system of claim 16, wherein the microservice deployment scheduler further comprises a rule-based dynamic workload orchestration module configured to orchestrate the operations of the one or more compute nodes using a predetermined set of rules associated with the microservice constraints and the node-specific attributes, wherein executing the instructions causes the processor to orchestrate the operations of the one or more compute nodes through the microservice deployment scheduler by inputting the microservice constraints and the node-specific attributes into the rule-based dynamic workload orchestration module, wherein if the predetermined set of rules is not associated with the microservice constraints and the node-specific attributes, the processor performs the operation of orchestrating the one or more compute nodes using the reinforcement learning-based dynamic workload orchestration module.

18. The computer system of claim 17, wherein the predetermined rule set implements a multi-objective problem statement.

19. The computer system according to any one of claims 16 to 18, wherein executing the instructions further causes the processor to train the reinforcement learning-based dynamic workload orchestration module using a reward function during operation, wherein the reward function includes any one of the following: resource utilization of the compute node, response time to the microservice constraints, cost efficiency of the compute node, energy consumption of the compute node, and combinations thereof.

20. The computer system according to any one of claims 16 to 19, wherein the distributed computing environment is configured to receive microservice applications from a microservice database for execution by the one or more computing nodes, wherein the microservice deployment scheduler is executed on a dedicated computing node, wherein the dedicated computing node serves as a gateway between the microservice database and the one or more computing nodes.

Images