By assessing microservices to container automation orchestration

Abstract

Container scaling and migration for container-based microservices are provided. A first set of features is extracted from each respective microservice of a plurality of different microservices. A number of containers required by each respective microservice of the plurality of different microservices at a future point in time is predicted using a trained prediction model and the first set of features extracted from each respective microservice. A scale tag and a scale value are assigned to each respective microservice of the plurality of different microservices based on a predicted change in a current number of containers corresponding to each respective microservice in accordance with the number of containers required by each respective microservice at the future point in time. The current number of containers corresponding to each respective microservice of the plurality of different microservices is adjusted based on the scale tag and the scale value assigned to each respective microservice.

Description

Technical Field

[0001] This disclosure generally relates to container-based microservices, and more specifically to automatically orchestrating containers for microservices by evaluating the intra-node and inter-node characteristics of the microservices and then evaluating the results of container orchestration. Background Technology

[0002] Container platforms are now used to wrap applications, allowing them to access a specific set of resources on the host operating system. In a microservices architecture, applications are further broken down into various discrete services, each wrapped in a separate container. The advantage is that containers are scalable and ephemeral. In other words, instances of applications or services hosted in containers can be moved around as needed.

[0003] However, scalability presents an operational challenge. Container orchestration is all about managing the lifecycle of containers, especially in large, dynamic environments. Container orchestration control automates many tasks, such as container provisioning and deployment, adding or removing containers to distribute application load evenly across host infrastructure, migrating containers from one host to another if resources are scarce or if a host crashes, and allocating resources among containers.

[0004] When it's time to deploy a new container to the cluster, the container orchestration tool schedules the deployment based on predefined constraints (such as the availability of processors, memory, storage, network resources, etc.) and finds the most suitable host to place the container. Containers can also be placed based on their proximity to other hosts.

[0005] A cluster is a group of nodes, which can be physical or virtual, consisting of at least one controller node and several worker nodes. Each node has its own operating system environment. The controller manages the scheduling and deployment of application instances on the nodes, and the complete set of services running on the controller node is called the control plane. The scheduler assigns nodes to pods based on resources and defined policy constraints. A pod is the basic unit of scheduling, consisting of one or more containers that are guaranteed to reside on the same host and can share resources. A unique IP address is assigned to each pod within the cluster, allowing applications to use ports without conflict.

[0006] Container orchestration tools (such as Kubernetes, Docker Swarm, etc.) are components used to automatically deploy, scale, and manage containerized applications across node clusters. Container orchestration tools group the containers that make up an application into logical units for easier management and discovery. Multi-cluster container orchestration environments can also manage clusters of containerized applications that can span public, private, and hybrid clouds.

[0007] Microservices are a group of pods that work together, such as a layer in a multi-tiered application. Microservices are an architecture and organizational method for software development where software comprises small, independent services that communicate through well-defined application programming interfaces (APIs). Microservice architecture makes applications easier to scale and develop faster, enabling innovation and accelerating time-to-market for new features. With microservice architecture, applications are built as independent components, each running as a service in its own process. These services communicate using lightweight APIs via well-defined interfaces. Services are built for business capabilities, and each service performs a single function. Because services run independently, each service can be updated, deployed, and scaled to meet the application's specific functionalities. Summary of the Invention

[0008] According to one exemplary embodiment, a computer-implemented method for automatically performing container scaling and migration of container-based microservices is provided. The computer extracts a first set of features from each corresponding microservice among a plurality of different microservices. The computer uses a trained prediction model and the first set of features extracted from each corresponding microservice to predict the number of containers required for each corresponding microservice among the plurality of different microservices at a future time point. The computer assigns a scaling label and a scaling value to each corresponding microservice among the plurality of different microservices based on the predicted change in the number of containers required for each corresponding microservice at the future time point, based on the current number of containers corresponding to each corresponding microservice. The computer automatically adjusts the current number of containers corresponding to each corresponding microservice among the plurality of different microservices based on the scaling label and scaling value assigned to each corresponding microservice. According to other exemplary embodiments, a computer system and computer program product for automatically performing container scaling and migration of container-based microservices are provided. Attached Figure Description

[0009] Figure 1 It is a graphical representation of a network in which exemplary embodiments of a data processing system can be implemented;

[0010] Figure 2 This is a diagram of a data processing system in which exemplary embodiments can be implemented;

[0011] Figure 3 This is a diagram illustrating a cloud computing environment in which exemplary embodiments can be implemented;

[0012] Figure 4 This is a diagram illustrating an example of an abstraction layer of a cloud computing environment according to an exemplary embodiment;

[0013] Figure 5 This is a diagram illustrating an example of a container orchestration system according to an exemplary embodiment;

[0014] Figure 6This is a diagram illustrating an example of a prediction table according to an exemplary embodiment;

[0015] Figure 7 This is a diagram illustrating an example of a container migration process according to an exemplary embodiment;

[0016] Figure 8 This is a diagram illustrating an example of a container migration identifier table according to an exemplary embodiment; and

[0017] Figures 9A-9B This is a flowchart illustrating a process for predicting container scaling and migration of container-based microservices according to an exemplary embodiment. Detailed Implementation

[0018] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0019] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0020] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0021] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing the status information of the computer-readable program instructions.

[0022] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0023] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0024] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0025] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a non-linear order as shown in the figures. For example, two blocks shown consecutively may actually be implemented as a single step, executed simultaneously, substantially simultaneously, with partial or complete time overlap, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0026] Now refer to the attached diagram, and in particular, refer to... Figure 1-5 A diagram is provided illustrating a data processing environment in which exemplary embodiments can be implemented. It should be understood that... Figure 1-5 This is merely an example and is not intended to assert or imply any limitation regarding the environments in which different embodiments may be implemented. Many modifications can be made to the described environment.

[0027] Figure 1A graphical representation of a network in which exemplary embodiments of a data processing system may be implemented is described. Network data processing system 100 is a network of computers, data processing systems, and other devices in which exemplary embodiments may be implemented. Network data processing system 100 includes network 102, which is a medium for providing communication links between computers, data processing systems, and other devices connected together within network data processing system 100. Network 102 may include connections such as, for example, wired communication links, wireless communication links, fiber optic cables, etc.

[0028] In the described example, servers 104 and 106, along with storage 108, are connected to network 102. Servers 104 and 106 may be, for example, server computers with a high-speed connection to network 102. Furthermore, servers 104 and 106 provide container orchestration services for microservices running on client compute node devices by evaluating the intra-node and inter-node characteristics of microservices and then evaluating the results of container orchestration (i.e., scaling up or down containers within compute nodes based on intra-node characteristic evaluation, and migrating containers between nodes based on inter-node characteristic evaluation). Additionally, it should be noted that servers 104 and 106 may each represent multiple servers in one or more cloud environments. Alternatively, servers 104 and 106 may each represent server clusters in one or more data centers.

[0029] Clients 110, 112, and 114 are also connected to network 102. Clients 110, 112, and 114 are client computing node devices for servers 104 and 106. In this example, clients 110, 112, and 114 are network computers with wired communication links to network 102. However, it should be noted that clients 110, 112, and 114 could represent other types of data processing systems with wired or wireless communication links to network 102, such as, for example, desktop computers, laptop computers, handheld computers, smartphones, smart vehicles, smart TVs, smart appliances, etc. Users can utilize these client devices to view the impact of container orchestration performed by servers 104 and 106 via a key performance indicator dashboard.

[0030] Storage 108 is a network storage device capable of storing any type of data in structured or unstructured formats. Furthermore, storage device 108 can represent multiple network storage devices. Additionally, storage 108 can store identifiers and network addresses of multiple different client computing node devices, identifiers of multiple different microservices, identifiers of multiple containers, intra-node characteristic data, inter-node characteristic data, information about the impact of container orchestration operations, historical microservice workload data for defined time periods, etc. Furthermore, storage 108 can store other types of data, such as authentication or credential data, which may include, for example, usernames, passwords, and biometric data associated with system administrators and users.

[0031] Additionally, it should be noted that the network data processing system 100 may include any number of additional servers, clients, storage devices, and other devices not shown. Program code located in the network data processing system 100 may be stored on a computer-readable storage medium and downloaded to a computer or other data processing device for use. For example, program code may be stored on a computer-readable storage medium on server 104 and downloaded to client 110 via network 102 for use on client 110.

[0032] In the described example, the network data processing system 100 can be implemented as many different types of communication networks, such as, for example, the Internet, intranet, wide area network (WAN), local area network (LAN), telecommunications network, or any combination thereof. Figure 1 This is intended only as an example and not as an architectural limitation on different exemplary embodiments.

[0033] As used in this article, when referring to projects, "several" means one or more projects. For example, "several different types of communication networks" refers to one or more different types of communication networks. Similarly, when referring to projects, "a group" means one or more projects.

[0034] Furthermore, the term "at least one" when used with a list of items means that different combinations of one or more items from the listed items may be used, and it is possible that only one item from each item in the list is required. In other words, "at least one" refers to any combination of items, and any number of items from the list can be used, but not all items in the list are required. Items can be specific objects, things, or categories.

[0035] For example, but not limited to, "at least one of project A, project B, or project C" can include project A, project A and project B, or project B. The example can also include project A, project B, and project C, or project B and project C. In some illustrative examples, "at least one" can be, for example, but not limited to, 2 projects A; 1 project B; and 10 projects C; 4 projects B and 7 projects C; or other suitable combinations.

[0036] Now for reference Figure 2 A diagram of a data processing system is depicted according to an exemplary embodiment. The data processing system 200 is, for example,... Figure 1 The example of a computer as server 104 includes computer-readable program code or instructions that implement a container orchestration process of an exemplary embodiment. In this example, data processing system 200 includes a communication structure 202 that provides communication between processor unit 204, memory 206, persistent storage 208, communication unit 210, input / output (I / O) unit 212, and display 214.

[0037] Processor unit 204 is used to execute instructions for software applications and programs that can be loaded into memory 206. Processor unit 204 may be a collection of one or more hardware processor devices, or it may be a multi-core processor, depending on the specific implementation.

[0038] Memory 206 and persistent storage 208 are examples of storage device 216. As used herein, a computer-readable storage device or computer-readable storage medium is any hardware capable of storing information such as, for example, but not limited to, data, computer-readable program code in a functional form, and / or other suitable information based on transient or persistent conditions. Furthermore, a computer-readable storage device or computer-readable storage medium does not include propagation media such as transient signals. In these examples, memory 206 may be, for example, random access memory (RAM), or any other suitable volatile or non-volatile storage device, such as flash memory. Persistent storage 208 may take various forms depending on the specific implementation. For example, persistent storage 208 may include one or more devices. For example, persistent storage 208 may be a disk drive, a solid-state drive, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The medium used in persistent storage 208 may be removable. For example, a removable hard disk drive may be used for persistent storage 208.

[0039] In this example, persistent storage 208 stores machine learning component 218. However, it should be noted that even though machine learning component 218 is shown residing in persistent storage 208, in alternative exemplary embodiments, machine learning component 218 may be a separate component of data processing system 200. For example, machine learning component 218 may be a hardware component coupled to communication structure 202 or a combination of hardware and software components. In another alternative exemplary embodiment, a first set of components of machine learning component 218 may reside in data processing system 200, and a second set of components of machine learning component 218 may reside in a second data processing system, such as, for example... Figure 1 Server 106 in the middle.

[0040] Machine learning component 218 controls the process of automatically orchestrating containers for multiple different microservices by evaluating intra-node and inter-node features of multiple different microservices and then evaluating the results of container orchestration. Machine learning component 218 can learn without being explicitly programmed to do so. Machine learning component 218 can learn based on training data input into machine learning component 218. Machine learning component 218 can learn using various types of machine learning algorithms. These various types of machine learning algorithms include at least one of supervised learning, semi-supervised learning, unsupervised learning, feature learning, sparse dictionary learning, anomaly detection, association rule processing, or other types of learning algorithms. Examples of machine learning models include artificial neural networks, decision trees, support vector machines, Bayesian networks, genetic algorithms, and other types of models. For example, these machine learning models can be trained using historical microservice workload data.

[0041] Microservice 220 represents the identifier of a specific container-based microservice. However, it should be noted that microservice 220 can represent the identifiers of multiple different microservices, against which machine learning component 218 performs container orchestration services (such as container scaling and container migration). Microservice 220 runs on container 224 located on compute node 222. Container 224 represents the identifiers of multiple containers located on each compute node of compute node 222. Compute node 222 represents multiple compute nodes (e.g., ...). Figure 1 The identifiers of clients 110, 112, and 114 in the container 224, and the compute node 222 provides resources (such as processors, memory, storage, network devices, etc.) for the container 224 to run microservices 220.

[0042] Machine learning component 218 utilizes microservice feature extractor component 226 to extract features (e.g., characteristics, attributes, performance, traits, parameters, etc.) corresponding to each corresponding compute node of compute node 222. The extracted features include intra-node features 228 and inter-node features 230.

[0043] The node features 228 include information such as the number of containers corresponding to the microservices 220 running on compute node 222 during the current time period, the utilization and workload capacity of the number of containers corresponding to the microservices 220 running on compute node 222 during the current time period, the number of containers corresponding to the microservices 220 previously running on compute node 222 during a previous time period, the utilization and workload capacity of the number of containers corresponding to the microservices 220 running on compute node 222 during a previous time period, the number of application programming interface requests to microservices 220 during the current time period, and the number of application programming interface requests to microservices 220 during a previous time period. Inter-node characteristics 230 include dependencies between microservice 220 and other microservices (e.g., microservice 220 makes application programming interface calls to one or more other microservices among a number of different microservices), relationships between microservice 220 and other microservices (e.g., microservice 220 and one or more other microservices use the same application or correspond to the same application), the geographic location of container 224 (e.g., the geographic location of each specific compute node running container 224 corresponding to microservice 220), and information on latency parameters and network bandwidth corresponding to each compute node 222.

[0044] Machine learning component 218 analyzes the extracted intra-node features 228 and inter-node features 230 to determine and generate optimal container orchestration strategies, such as container scaling within compute nodes 222 (i.e., scaling up or down containers) and container migration between certain compute nodes 222, to improve microservice performance and reduce network latency. Machine learning component 218 guides orchestration component 232 to implement the determined optimal container orchestration strategy.

[0045] Furthermore, machine learning component 218 determines the impact of container orchestration on a set of key performance indicators (KPIs), such as, for example, the cost of container scaling and migration, network latency, and microservice security. Machine learning component 218 utilizes key performance indicator dashboard component 234 to generate and display the results of the impact of container orchestration on the set of KPIs. Key performance indicator dashboard component 234 displays the results of the impact to the user via a dashboard (e.g., an interactive graphical user interface) on a display device such as display 214.

[0046] As a result, the data processing system 200 operates as a dedicated computer system, wherein the machine learning component 218 within the data processing system 200 enables the automatic orchestration of containers to improve microservice performance and reduce network latency. Specifically, the machine learning component 218 transforms the data processing system 200 into a dedicated computer system compared to currently available general-purpose computer systems that do not possess the machine learning component 218.

[0047] In this example, communication unit 210 is connected via a network (e.g., Figure 1 Network 102) provides communication with other computers, data processing systems, and devices. Communication unit 210 can provide communication using physical and wireless communication links. The physical communication link can be established for the data processing system 200 using, for example, wires, cables, universal serial buses, or any other physical technology. The wireless communication link can utilize, for example, shortwave, high frequency, ultra-high frequency, microwave, Wi-Fi, etc. The technology, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), 2G, 3G, 4G, 4G LTE, LTE Advanced, 5G, or any other wireless communication technology or standard is used to establish a wireless communication link for the data processing system 200.

[0048] Input / output unit 212 allows data input and output to other devices that can be connected to data processing system 200. For example, input / output unit 212 can provide a connection for user input via a keyboard, keypad, mouse, microphone, and / or some other suitable input device. Display 214 provides a mechanism for displaying information to the user and may include touchscreen capability to allow the user to make on-screen selections, for example, through a user interface or input data.

[0049] Instructions for operating systems, applications, and / or programs may reside in storage device 216, which communicates with processor unit 204 via communication structure 202. In this illustrative example, the instructions reside functionally on persistent storage 208. These instructions may be loaded into memory 206 for execution by processor unit 204. Processes in different embodiments may be executed by processor unit 204 using computer-implemented instructions, which may reside in memory such as memory 206. These program instructions are referred to as program code, computer-usable program code, or computer-readable program code, and may be read and executed by a processor in processor unit 204. In different embodiments, program instructions may be embodied on different physical computer-readable storage devices, such as memory 206 or persistent storage 208.

[0050] Program code 236 is functionally located on computer-readable medium 238, which can be selectively removed and loaded into or transferred to data processing system 200 for execution by processor unit 204. Program code 236 and computer-readable medium 238 form computer program product 240. In one example, computer-readable medium 238 may be computer-readable storage medium 242 or computer-readable signal medium 244.

[0051] In these illustrative examples, computer-readable storage medium 242 is a physical or tangible storage device for storing program code 236, rather than a medium for disseminating or transmitting program code 236. Computer-readable storage medium 242 may include, for example, an optical disc or disk, which is inserted into or placed into a drive or other device that is part of persistent storage 208 for transfer to a storage device (such as a hard disk drive) that is part of persistent storage 208. Computer-readable storage medium 242 may also take the form of persistent storage, such as a hard disk drive, thumb drive, or flash memory connected to data processing system 200.

[0052] Alternatively, program code 236 may be transmitted to data processing system 200 using computer-readable signal medium 244. Computer-readable signal medium 244 may be, for example, a propagated data signal containing program code 236. For example, computer-readable signal medium 244 may be an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted via a communication link, such as a wireless communication link, fiber optic cable, coaxial cable, wire, or any other suitable type of communication link.

[0053] Furthermore, as used herein, "computer-readable medium 238" can be singular or plural. For example, program code 236 may reside in a single storage device or system-type computer-readable medium 238. In another example, program code 236 may reside in computer-readable medium 238 distributed across multiple data processing systems. In other words, some instructions in program code 236 may reside in one data processing system, while other instructions in program code 236 may reside in one or more other data processing systems. For example, a portion of program code 236 may reside in a computer-readable medium 238 in a server computer, while another portion of program code 236 may reside in a computer-readable medium 238 in a set of client computers.

[0054] The different components shown for data processing system 200 do not imply any architectural limitation on how different embodiments can be implemented. In some illustrative examples, one or more components may be incorporated into or otherwise formed part of another component. For example, in some illustrative examples, memory 206 or a portion thereof may be incorporated into processor unit 204. Different exemplary embodiments may be implemented in data processing systems that include, or replace, those components shown for data processing system 200. Figure 2 The other components shown may differ from the illustrative example shown. Different embodiments can be implemented using any hardware device or system capable of running program code 236.

[0055] In another example, a bus system can be used to implement communication structure 202 and can include one or more buses, such as a system bus or input / output bus. Of course, the bus system can be implemented using any suitable type of architecture that provides data transfer between different components or devices attached to the bus system.

[0056] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Rather, exemplary embodiments can be implemented in conjunction with any other type of computing environment now known or developed hereafter. Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources, such as, for example, networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services, which can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0057] This feature can include, for example, on-demand self-service, broad network access, resource pooling, rapid elasticity, and metric services. On-demand self-service allows cloud consumers to unilaterally and automatically provision computing power, such as server time and network storage, on demand without human interaction with the service provider. Broad network access provides the ability to be available on the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms such as mobile phones, laptops, and PDAs. Resource pooling allows the pooling of a provider's computing resources to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated on demand. Location independence has significance because consumers typically do not control or know the exact location of the resources being provided, but can specify the location at a higher level of abstraction, such as country, state, or data center. Rapid elasticity provides the ability to be provided quickly and elastically, automatically scaling out and in rapidly in some cases. For consumers, the capacity available for provision often appears unlimited and can be purchased in any quantity at any time. Measurement services allow cloud systems to automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to service types such as storage, processing, bandwidth, and active user accounts. Resource usage can be monitored, controlled, and reported, providing transparency for both service providers and consumers.

[0058] Service models can include, for example, Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). SaaS provides consumers with the ability to use a provider's applications running on cloud infrastructure. Applications can be accessed from various client devices via thin client interfaces, such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with possible exceptions of limited user-specific application configuration settings. Platform as a Service provides consumers with the ability to deploy consumer-created or acquired applications onto cloud infrastructure, using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but have control over the deployed applications and, possibly, the configuration of the application hosting environment. Infrastructure as a Service provides consumers with the processing, storage, networking, and other basic computing resources to deploy and run arbitrary software, which can include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components such as host firewalls.

[0059] Deployment models can include, for example, private clouds, community clouds, public clouds, and hybrid clouds. A private cloud is cloud infrastructure operated solely by an organization. Private clouds can be managed by the organization or a third party and can exist on-site or off-site. A community cloud is cloud infrastructure shared by several organizations and supports a specific community with shared concerns, such as mission, security requirements, policies, and compliance considerations. Community clouds can be managed by the organization or a third party and can exist on-site or off-site. A public cloud is cloud infrastructure available to the general public or large industrial groups and is owned by an organization that sells cloud services. A hybrid cloud is a cloud infrastructure consisting of two or more clouds, such as, for example, private clouds, community clouds, and public clouds, which remain a single entity but are bound together by standardized or proprietary technologies that enable data and application portability, such as cloud bursting for load balancing between clouds.

[0060] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0061] Now for reference Figure 3The diagram illustrates a cloud computing environment in which exemplary embodiments can be implemented. In this illustrative example, the cloud computing environment 300 includes a collection of one or more cloud computing nodes 310 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants or smartphones 320A, desktop computers 320B, laptop computers 320C, and / or automotive computer systems 320N. Figure 1 Servers 104 and 106 are mentioned. Local computing devices 320A-320N can be, for example... Figure 1 Clients 110-114 in the middle.

[0062] Cloud computing nodes 310 can communicate with each other and can be physically or virtually grouped into one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds or combinations thereof as described above. This allows cloud computing environment 300 to provide infrastructure, platform, and / or software as a service for which cloud consumers do not need to maintain resources on local computing devices such as local computing devices 320A-320N. It should be understood that the types of local computing devices 320A-320N are for illustrative purposes only, and cloud computing nodes 310 and cloud computing environment 300 can communicate with any type of computerized device, for example, using a web browser via any type of network and / or network-addressable connection.

[0063] Now for reference Figure 4 A diagram illustrating abstract model layers is depicted according to an exemplary embodiment. The set of functional abstract layers shown in this illustrative example can be provided by a cloud computing environment, such as... Figure 3 The cloud computing environment in 300. It should be understood in advance that, Figure 4 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As described, the following layers and corresponding functions are provided.

[0064] The abstraction layer of the cloud computing environment 400 includes a hardware and software layer 402, a virtualization layer 404, a management layer 406, and a workload layer 408. The hardware and software layer 402 includes the hardware and software components of the cloud computing environment. Hardware components may include, for example, a mainframe 410, servers 412 and 414 based on a RISC (Reduced Instruction Set Computer) architecture, blade servers 416, storage devices 418, and network and networking components 420. In some exemplary embodiments, software components may include, for example, network application server software 422 and database software 424.

[0065] The virtualization layer 404 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 426; virtual storage 428; virtual network 430, including virtual private network; virtual application and operating system 432; and virtual client 434.

[0066] In one example, management layer 406 can provide the functions described below. Resource provisioning 436 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 438 provides cost tracking as resources are utilized within the cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 440 provides access to the cloud computing environment for consumers and system administrators. Service level management 442 provides cloud resource allocation and management to ensure that required service levels are met. Service level agreement (SLA) planning and fulfillment 444 provides pre-scheduling and procurement of cloud resources for anticipated future needs in accordance with the SLA.

[0067] Workload layer 408 provides examples of functionalities that can be leveraged in a cloud computing environment. Example workloads and functionalities that may be provided by workload layer 408 may include mapping and navigation 446, software development and lifecycle management 448, virtual classroom education delivery 450, data analytics processing 452, transaction processing 454, and microservice container orchestration management 456.

[0068] Computing is now concentrated on cloud platforms, and most entities, such as companies, businesses, enterprises, organizations, institutions, agencies, etc., provide their provisioning as microservices on cloud infrastructure. These provisionings are best practice for deploying microservices hosted within containers. However, deployment is performed by an orchestrator. Current orchestrators stipulate unidirectional provisioning, lacking autoscaling and optimized proximity alignment for microservice containers. An exemplary embodiment utilizes a machine learning component with the intelligence to inform the orchestrator about autoscaling and optimal proximity alignment of microservice containers based on a user-configurable context. In other words, the exemplary embodiment provides the orchestrator with intelligence for autoscaling and optimized migration of containers between compute nodes, and then evaluates autoscaling and migration interventions.

[0069] Current container orchestration tools (such as Kubernetes, Docker Swarm, etc.) are driven by static, user-defined policies and are not intelligent enough to make efficient container scaling decisions or optimal container migrations. Exemplary embodiments take into account that these current container orchestration tools perform container scaling up or down at runtime without anticipating future loads and do not perform automatic container migration to reduce network latency. To address these issues, exemplary embodiments use machine learning to predict container scaling and optimal container migrations, and then evaluate these interventions. The machine learning component of exemplary embodiments analyzes and evaluates the intra-node and inter-node characteristics of microservices to automatically determine efficient container orchestration.

[0070] For example, exemplary embodiments perform synchronized and optimized orchestration of automatic scaling of containers on compute nodes via proactive prediction, automatic migration of containers based on microservice similarity analysis, and automatic evaluation of interventions by machine learning components for container scaling and migration. Exemplary embodiments extract and analyze intra-node and inter-node features corresponding to microservices. Exemplary embodiments utilize the extracted intra-node features to proactively predict the number of containers required to meet future microservice loads (i.e., the number of containers to scale). Exemplary embodiments utilize the extracted inter-node features to evaluate the similarity between microservices to identify which containers should be migrated to reduce network latency.

[0071] Therefore, exemplary embodiments provide one or more technical solutions that overcome technical challenges by automatically performing container orchestration to meet anticipated future microservice workload demands. Thus, these one or more technical solutions provide both technical effectiveness and practical application in the realm of container-based microservices.

[0072] Now for reference Figure 5 The diagram illustrates an example of a container orchestration system according to an exemplary embodiment. The container orchestration system 500 can be used in, for example... Figure 1 The network data processing system 100 is implemented in the network of the data processing system. The container orchestration system 500 is a system for automatically orchestrating the hardware and software components of containers for multiple different microservices by evaluating the intra-node and inter-node characteristics of multiple different microservices and then evaluating the results of container orchestration.

[0073] In this example, container orchestration system 500 includes machine learning component 502, microservice feature extractor component 504, orchestrator component 506, key performance indicator dashboard component 508, compute node 510, and compute node 512. However, it should be noted that container orchestration system 500 is intended only as an example and not as a limitation on the exemplary embodiments. In other words, container orchestration system 500 may include more or fewer components than those shown. For example, one or more components may be combined into one component, components may be divided into two or more components, components not shown may be added, and so on. Furthermore, compute node 510 and compute node 512 may each represent multiple compute nodes.

[0074] Compute nodes 510 and 512 provide the actual infrastructure for host containers 516, 518, 520, 522, 524, 526, 528, and 530. Compute nodes 510 and 512 run microservice 514. Microservice 514 is a container-based microservice. Moreover, microservice 514 can represent a corresponding microservice among several different microservices managed by machine learning component 502.

[0075] Microservices 514 are loosely coupled services focused on performing a single business task and can scale both horizontally and vertically. This means that microservices 514 can provide efficient computation regarding workload management. Microservices 514 are also fault-tolerant and self-healing (i.e., perform fault management). Furthermore, microservices 514 are capable of both batch and real-time processing and are responsive, resilient, and elastic. All these properties make microservices 514 an excellent candidate for high availability and portability across compute nodes.

[0076] In this example, compute node 510 uses containers 516 and 518 of pod 532 and containers 520 and 522 of pod 534 to run a portion of microservice 514. Compute node 512 uses containers 524 and 526 of pod 536 and containers 528 and 530 of pod 538 to run another portion of microservice 514.

[0077] Machine learning component 502 automatically determines the strategy for dynamic container orchestration corresponding to microservice 514. Machine learning component 502 utilizes information about in-node features (such as...) Figure 2 The machine learning component 502 uses data on intra-node features 228 to proactively predict the number of containers required within each of compute nodes 510 and 512 (i.e., scaling up or down the number of containers) for the predicted future workload of microservice 514. The machine learning component 502 also utilizes data on inter-node features (such as...) Figure 2The machine learning component 502 uses data from inter-node features 230 to migrate containers between compute nodes 510 and 512 based on the relationships and dependencies between microservice 514 and one or more other microservices among multiple different microservices, in order to reduce or minimize latency in the network. In this example, the machine learning component 502 includes a prediction module 549, a migration module 542, and an evaluator module 544.

[0078] Machine learning component 502 utilizes microservice information extractor component 504 to identify, collect, and extract intra-node feature data and inter-node feature data, representing all necessary information related to microservice 514. Intra-node feature data may include, for example, the number of containers running for microservice 514, the number of application programming interface requests used per defined time interval to determine the workload of microservice 514, and the utilization and workload capacity of each container of microservice 514 per defined time interval. Inter-node feature data may include, for example, the dependencies between microservice 514 and one or more other microservices when microservice 514 calls another microservice, the relationships between microservice 514 and one or more other microservices when the same application uses these specific microservices, the geographical locations of compute nodes 510 and 512 corresponding to containers 516-530 of microservice 514, the network bandwidth and latency of connections corresponding to compute nodes 510 and 512, and the cost and configuration of compute nodes 510 and 512.

[0079] Machine learning component 502 utilizes prediction module 540 to predict the scaling up and scaling down of containers within compute nodes 510 and 512 based on the predicted future workload of microservice 514. Prediction module 540 uses a time series prediction model based on in-node feature data to predict the required number of containers to match the predicted future workload of microservice 514. For example, prediction module 540 may utilize Autoregressive Integral Moving Average (ARIMA) as the prediction model. ARIMA is a way of modeling time series data for prediction (i.e., for predicting future points in a time series). An ARIMA model is a specific type of regression model in which the dependent variable is stabilized. The independent variables are all lags and / or lags of the error of the dependent variable, thus in principle directly extending the ARIMA model to incorporate information provided by the primary key performance indicators and other exogenous variables. Essentially, prediction module 540 adds one or more regressors to the following prediction equation:

[0080]

[0081] Where “Yt” equals the number of containers in the current time period, “Yt-1” equals the number of containers in the previous time period, “Xt” equals the number of Application Programming Interface (API) requests in the current time period, and “Xt-1” equals the number of API requests in the previous time period. The prediction equation above represents the current number of containers required in the current time period “t”, which is predicted as a function of the number of containers in the previous time period “t-1” and the number of API requests in the current time period “t” and the previous time period “t-1”. This prediction equation is a general implementation that allows for easy inclusion of new input parameters, such as, for example, the capacity, utilization, and hysteresis values ​​of previous time periods “t-2”, “t-3”, etc. Typical machine learning optimization algorithms (e.g., gradient descent) can be used to train the prediction model to improve prediction accuracy and reduce error.

[0082] The prediction module 540 uses the prediction model trained above to predict the number of containers required for microservice 514. The prediction module 540 identifies scaling labels and scaling values ​​by comparing them to the current number of containers. As an illustrative example, the current number of containers in compute node 510 is 4, and the predicted number of containers required for the predicted future workload is 7. Therefore, in this example, the scaling label is "UP," and the scaling value is 3 (i.e., scaling up the current number of 4 containers to equal the total of 7 containers in compute node 510 by adding 3 new containers to meet the predicted future workload of microservice 514). Similarly, the scaling label could be "DOWN," and the scaling value could be 1 (i.e., shrinking the current number of 4 containers to equal the total of 3 containers in compute node 510 by removing 1 container to meet the predicted future workload of microservice 514). Based on the scaling labels and values, the orchestrator component 506 removes containers or adds new containers as needed.

[0083] In other words, if the predicted container value is higher than the current container value, scaling up is required. Conversely, if the predicted container value is lower than the current container value, scaling down is required. It should be noted that the prediction module 540 repeats this prediction process for each corresponding microservice across multiple different microservices, assigning a corresponding container scaling label and value. Afterward, the prediction module 540 sends all scaling labels and values ​​to the orchestrator component 506, which removes unnecessary containers or creates new ones as needed. However, it should be noted that the scaling label can be "No" and the scaling value can be 0, indicating that the current number of containers on a particular compute node does not need to be changed. The prediction module 540 also calculates the cost associated with scaling, which will be used by the evaluator module 544.

[0084] After orchestrator component 506 performs scaling of the current number of containers based on the output of prediction module 540, machine learning component 502 utilizes migration module 542 to optimize container migration between compute nodes corresponding to microservice 514 and containers corresponding to multiple other microservices. In other words, migration module 542 identifies which containers need to be migrated and where within the network they should be migrated to (i.e., to which compute nodes). For example, migration module 542 identifies microservices with network latency greater than a defined network latency threshold. Migration module 542 also identifies those microservices that are most similar to each other (e.g., those microservices that have a defined similarity to each other based on extracted inter-node feature data). Migration module 542 then instructs orchestrator component 506 to migrate containers corresponding to microservices with the defined similarity to the same compute node to reduce network latency. Migration module 542 also calculates the costs associated with the migration and microservice security, which may also be used by evaluator module 544.

[0085] Orchestrator component 506 defines how to deploy, monitor, and configure containers using container orchestration strategies. During runtime, depending on the microservice workload, orchestrator component 506 scales up or down containers on compute nodes and migrates containers between compute nodes based on container orchestration strategies generated by and received from the prediction module 540 and migration module 542 of machine learning component 502.

[0086] The evaluator module 544 evaluates the container scaling and migration interventions performed by the prediction module 540 and the migration module 542. The evaluator module 544 may utilize, for example, causal inference regulation equations to measure the impact of each intervention, such as how container scaling affects cost and network latency, and how container migration affects cost, network latency, and microservice security. For example, E (Key Performance Indicator / Intervention) is the impact on key performance indicators (e.g., cost, latency, security, etc.) of the conditions under which a certain type of intervention (e.g., container scaling and / or container migration) is performed.

[0087] The evaluator module 544 uses the following two equations to evaluate the impact of the prediction module 540 on key performance metrics:

[0088] Cost increment = E(cost / before scaling) - E(cost / after scaling); and

[0089] The delay increment value = E(delay / before scaling) - E(delay / after scaling), where the cost and delay increment value are measured based on the prediction of the prediction module 540 to measure the impact of automatic container scaling.

[0090] The evaluator module 544 uses the following three equations to evaluate the impact of key performance indicators of the migration module 542:

[0091] Cost increment = E(cost / before migration) - E(cost / after migration);

[0092] Delay increment = E(delay / before migration) - E(delay / after migration); and

[0093] Security increment value = E(security / before migration) - E(security / after migration), where cost, latency and security increment values ​​are measured based on the microservice similarity analysis of migration module 542 to measure the impact of automatic container migration.

[0094] The Key Performance Indicator (KPI) dashboard component 508 generates and displays a KPI dashboard, which users can use to visualize the output generated by the evaluator module 544 (i.e., corresponding to different incremental values ​​from the prediction module 540 and migration module 542). As a result, users can monitor the impact of container orchestration interventions by the prediction module 540 and migration module 542 on a selected set of key performance indicators, such as container scaling and migration costs, network latency, and microservice security.

[0095] Now for reference Figure 6 A diagram illustrating an example prediction table is depicted according to an exemplary embodiment. Prediction table 600 can be implemented in a prediction module, such as, for example... Figure 5 The prediction module 540 is included. The prediction table 600 includes an x-axis timeline 602 and a container 604 for the number of y-axis axes.

[0096] Timeline 602 is a user-defined and adjustable time window. In other words, the unit of timeline 602 can be, for example, hours, days, weeks, months, etc., depending on the time window the user wants the forecasting module to analyze, in order to use a time series forecasting model to predict future microservice workloads. The number of containers 604 indicates the number of containers currently needed by the microservice up to unit "24" of timeline 602, and thereafter indicates the predicted number of containers required by the microservice (i.e., forecast 606). Forecast table 600 also shows the lower bound 608 and upper bound 610 of the forecast confidence level corresponding to forecast 606.

[0097] Now for reference Figure 7 The diagram illustrates an example of a container migration process according to an exemplary embodiment. The container migration process 700 can be implemented in a migration module, such as, for example... Figure 5 The migration module 542. In this example, the container migration process 700 is performed by the migration module between compute node A 702 and compute node B 704. However, it should be noted that the migration module can perform the container migration process 700 between any number of compute nodes.

[0098] In this example as well, compute node A 702 includes containers 706 and 708, and compute node B 704 includes containers 710 and 712. However, it should be noted that compute node A 702 and compute node B 704 can include any number of containers. Furthermore, in this example, application 1 714 uses container 706, while application 716 uses containers 708 and 712.

[0099] Furthermore, in this example, a latency bottleneck 718 exists between containers 706 and 710. As a result, the migration module determines that container 710 needs to be migrated from compute node B 704 to compute node A 702 to reduce network latency caused by the latency bottleneck 718. Therefore, the migration module guides the orchestrator components (e.g., Figure 5 The orchestrator component 506 in the middle performs the migration 720 of container 710 to compute node A 702. Furthermore, a latency bottleneck 722 exists between container 708 and application 2 716. As a result, the migration module determines that container 708 needs to be migrated from compute node A 702 to compute node B 704 to reduce network latency caused by the latency bottleneck 722. Therefore, the migration module instructs the orchestrator component to perform the migration 724 of container 708 to compute node B 704.

[0100] Now for reference Figure 8 A diagram illustrating an example of a container migration identifier table is depicted according to an exemplary embodiment. The container migration identifier table 800 may be implemented in a migration module, for example... Figure 5 Migration module 542.

[0101] In this example, the container migration identification table 800 includes microservices 802, dependencies 804, relationships 806, compute nodes 808, network latency 810, and shared data attributes 812. Microservices 802 identify each corresponding microservice. Dependencies 804 identify dependencies between certain microservices. Relationships 806 identify relationships between certain microservices based on the application. In other words, different microservices are connected at the application level. Compute nodes 808 identify specific compute nodes in the network associated with a particular microservice. Network latency 810 identifies the amount of network latency associated with each specific microservice. Shared data attributes 812 identify information accessed by a specific microservice. Data sharing between different microservices is accommodated via a shared database containing both static and mutable data, and different microservices access the required data from the shared database.

[0102] In this example, container migration identification table 800 shows three microservices: MS-A, MS-B, and MS-X. Container migration identification table 800 also shows their respective dependencies, applications, nodes, and network latency. The migration module uses dependency 804, application information in relationship 806, and shared data attribute 812 information about accessing and retrieving the corresponding microservices to identify the similarity between two microservices.

[0103] The migration module can use similarity calculations (such as cosine similarity) to determine microservice similarity. In this example, the migration module calculates the similarity between microservice A (MS-A) and microservice B (MS-B) as similarity(MS-A, MS-B) = 0.95. In other words, MS-A and MS-B have 95% similarity, which is indicated as "similar" on the container migration identifier table 800. Furthermore, the migration module calculates the similarity between MS-B and microservice X (MS-X) as similarity(MS-B, MS-X) = 0.30. In other words, MS-B and MS-X have only 30% similarity. The container migration identifier table 800 also indicates that MS-A should be migrated from node A to node C to reduce network latency.

[0104] Now for reference Figures 9A-9B The following is a flowchart illustrating a process for predicting container scaling and migration of container-based microservices, according to an exemplary embodiment. Figures 9A-9B The process shown can be performed in, for example, Figure 1 Server 104 or Figure 2 The data processing system 200 is implemented in the computer. For example, Figures 9A-9B The process shown can be performed in Figure 2 It is implemented in the machine learning component 218.

[0105] The process begins with the computer training a prediction model to form a trained prediction model that predicts the scaling of multiple containers corresponding to each of multiple different microservices running on multiple compute nodes in the network, based on the historical number of containers required to satisfy the workload of each corresponding microservice during a defined time period (step 902). The computer extracts a first set of features from each of the multiple different microservices (step 904). The first set of features is selected from the group including: a first number of containers corresponding to each of the multiple different microservices running on multiple compute nodes during the current time period; at least one of the utilization and workload capacity of the first number of containers corresponding to each of the multiple different microservices running on multiple compute nodes during the current time period; a second number of containers corresponding to each of the multiple different microservices previously running on multiple compute nodes during a previous time period; at least one of the utilization and workload capacity of the second number of containers corresponding to each of the multiple different microservices running on multiple compute nodes during a previous time period; a first number of application programming interface (API) requests for each of the multiple different microservices running on multiple compute nodes during the current time period; and a second number of API requests for each of the multiple different microservices running on multiple compute nodes during the current time period.

[0106] The computer uses a trained prediction model and a first set of features extracted from each corresponding microservice to predict the number of containers required by each of the multiple different microservices at a future time point (step 906). Based on the predicted change in the current number of containers corresponding to each corresponding microservice at the future time point, the computer assigns scaling labels and scaling values ​​(e.g., scaling up or down by values ​​such as one, two, three, etc.) to each of the multiple different microservices based on the scaling labels and scaling values ​​assigned to each corresponding microservice (step 908). Based on the scaling labels and scaling values ​​assigned to each corresponding microservice, the computer automatically adjusts the current number of containers corresponding to each of the multiple different microservices (step 910). Adjusting the current number of containers includes: not adjusting the current number of containers; generating one or more additional containers for a specific microservice with assigned scaling labels and scaling values ​​indicating that they need to be increased at a future time point; and removing one or more current containers for a specific microservice with assigned scaling labels and scaling values ​​indicating that they need to be reduced at a future time point.

[0107] The computer uses causal reasoning to determine the first impact of adjusting the current number of containers corresponding to each of the multiple different microservices on the container scaling costs and network latency of these compute nodes (e.g., compute nodes with an increased number of containers and compute nodes with a reduced number of containers) with respect to the adjusted number of containers (step 912). The computer extracts a second set of features from each of the multiple different microservices (step 914). The second set of features is selected from a group including: information about dependencies between specific microservices (e.g., which specific microservices make application programming interface calls to other microservices among the multiple different microservices); information about relationships between specific microservices (e.g., which applications use the same microservices); information about the geographic location corresponding to each of the multiple containers corresponding to each of the multiple different microservices (e.g., where each specific compute node running multiple containers corresponding to the same microservice among the multiple different microservices is geographically located); and network bandwidth and latency parameters corresponding to each of the multiple compute nodes. The computer determines the degree of microservice similarity among specific microservices in a plurality of different microservices based on a second set of features extracted from each corresponding microservice (step 916).

[0108] The computer identifies a first group of containers running on a first group of compute nodes across a plurality of compute nodes, having network latency values ​​higher than a network latency threshold level (step 918). Based on the degree of microservice similarity determined among certain microservices, the computer identifies a second group of compute nodes running a second group of containers, which has a container similarity degree higher than a container similarity threshold level with the first group of containers running on the first group of compute nodes, and which has a network latency value higher than a network latency threshold level with the first group of containers running on the first group of compute nodes (step 920). The computer identifies specific containers within the first and second groups of containers that share the same container similarity degree (step 922).

[0109] The computer migrates specific containers that share the same degree of container similarity to the same compute node to reduce network latency (step 924). The computer uses causal reasoning to determine a second impact of migrating these specific containers between the two compute nodes on a second set of key performance indicators (KPIs) regarding container migration costs, network latency, and microservice security (step 926). The computer displays, within the KPIs dashboard, the first impact of adjusting the current number of containers corresponding to each of the multiple different microservices on the first set of KPIs and the second impact of migrating these specific containers between the two compute nodes on the second set of KPIs (step 928). The process then terminates.

[0110] Therefore, exemplary embodiments of the present invention provide a computer-implemented method, computer system, and computer program product for automatically orchestrating containers by evaluating intra-node and inter-node characteristics of microservices and then evaluating the results of container orchestration. Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method for automatically performing container scaling and migration of container-based microservices, the computer-implemented method comprising: The computer extracts the first set of features from each of the corresponding microservices among multiple different microservices; The computer uses a trained prediction model and the first set of features extracted from each corresponding microservice to predict the number of containers required for each of the multiple different microservices at a future time point. The computer assigns scaling labels and scaling values ​​to each of the plurality of different microservices based on a predicted change in the number of containers required by each corresponding microservice at a future point in time, based on the current number of containers corresponding to each corresponding microservice. The computer automatically adjusts the current number of containers corresponding to each of the plurality of different microservices based on the scaling label and the scaling value assigned to each corresponding microservice; The computer extracts a second set of features from each of the multiple different microservices; as well as The computer determines the degree of microservice similarity among specific microservices in the plurality of different microservices based on the second set of features extracted from each corresponding microservice.

2. The computer-implemented method according to claim 1 further includes: The computer identifies a first group of containers running on a first group of computing nodes among a plurality of computing nodes, which have network latency values ​​higher than a network latency threshold level. as well as The computer determines a second set of compute nodes running a second set of containers among the plurality of compute nodes based on the determined degree of similarity between the microservices in the specific microservices. The second set of containers has a container similarity degree higher than the container similarity threshold level of the first set of containers running on the first set of compute nodes with a network latency value higher than the network latency threshold level.

3. The computer-implemented method according to claim 2 further includes: The computer identifies specific containers in the first group of containers and the second group of containers that share the same degree of container similarity. as well as The computer migrates specific containers that share the same degree of container similarity to the same computing node to reduce network latency.

4. The computer-implemented method according to claim 3 further includes: The computer uses causal inference to determine the first impact of adjusting the current number of containers corresponding to each of the plurality of different microservices on a first set of key performance indicators of these compute nodes with respect to the adjusted number of containers among the plurality of compute nodes; The computer uses the causal inference adjustment to determine the second impact of migrating these specific containers between two computing nodes on a second set of key performance indicators regarding the two computing nodes. as well as The computer displays, within a key performance indicator dashboard, the first impact of adjusting the current number of containers corresponding to each of the plurality of different microservices on the first set of key performance indicators and the second impact of migrating these specific containers between two compute nodes on the second set of key performance indicators.

5. The computer-implemented method according to claim 1, further comprising: The computer trains a prediction model to predict the scaling of multiple containers corresponding to each of multiple different microservices running on multiple compute nodes in the network, based on the historical number of containers required to satisfy the workload of each corresponding microservice during a defined time period, in order to form the trained prediction model.

6. The computer-implemented method according to claim 5, wherein, The prediction model is an autoregressive integral moving average model.

7. The computer-implemented method according to claim 1, wherein, The first set of features is selected from a group including: a first number of containers corresponding to each of the plurality of different microservices running on the plurality of compute nodes during the current time period, at least one of the utilization and workload capacity of the first number of containers corresponding to each of the plurality of different microservices running on the plurality of compute nodes during the current time period, and a second number of containers corresponding to each of the plurality of different microservices previously running on the plurality of compute nodes during a previous time period. The utilization and workload capacity of at least one of the second number of containers corresponding to each of the multiple different microservices running on the multiple compute nodes during the previous time period, the first number of application programming interface (API) requests for each corresponding microservice during the current time period, and the second number of API requests for each corresponding microservice during the current time period.

8. The computer-implemented method according to claim 1, wherein, The second set of features is selected from a group that includes: information about dependencies between specific microservices, information about relationships between specific microservices, information about the geographic location of each of the multiple containers corresponding to each of the multiple different microservices, and network bandwidth and latency parameters corresponding to each of the multiple compute nodes.

9. The computer-implemented method according to claim 1, wherein, Adjusting the current number of containers includes one of the following: generating one or more additional containers for a specific microservice that has assigned scaling labels and scaling values ​​that indicate an increase is needed at the future time point, and removing one or more current containers for a specific microservice that has assigned scaling labels and scaling values ​​that indicate a decrease is needed at the future time point.

10. A computer system for automatically performing container scaling and migration of container-based microservices, the computer system comprising: Bus system; A storage device connected to the bus system, wherein the storage device stores program instructions; and A processor connected to the bus system, wherein the processor executes the program instructions to: Extract the first set of features from each of the multiple different microservices; The trained prediction model and the first set of features extracted from each corresponding microservice are used to predict the number of containers required for each of the multiple different microservices at a future time point. Based on the predicted change in the number of containers required by each corresponding microservice at a future time point, based on the current number of containers corresponding to each corresponding microservice, a scaling label and scaling value are assigned to each of the plurality of different microservices. Based on the scaling label and scaling value assigned to each corresponding microservice, the current number of containers corresponding to each of the plurality of different microservices is automatically adjusted; Extract a second set of features from each of the multiple different microservices; and The degree of microservice similarity among specific microservices in the plurality of different microservices is determined based on the second set of features extracted from each corresponding microservice.

11. The computer system according to claim 10, wherein, The processor further executes the program instructions to: Identify the first group of containers running on the first group of compute nodes in a multi-compute node cluster, which have network latency values ​​higher than a network latency threshold level. as well as Based on the determined degree of microservice similarity among the specific microservices, a second set of compute nodes is determined among the plurality of compute nodes to run a second set of containers, the second set of containers having a container similarity degree above a container similarity threshold level with the first set of containers running on the first set of compute nodes that have a network latency value higher than the network latency threshold level.

12. The computer system according to claim 11, wherein, The processor further executes the program instructions to: Identify specific containers in the first group of containers and the second group of containers that share the same degree of container similarity; as well as Migrate specific containers that share the same level of container similarity to the same compute node to reduce network latency.

13. The computer system according to claim 12, wherein, The processor further executes the program instructions to: Causal inference adjustment is used to determine the first impact of adjusting the current number of containers corresponding to each of the plurality of different microservices on a first set of key performance indicators of these compute nodes with respect to the adjusted number of containers among the plurality of compute nodes; The causal inference adjustment is used to determine the second impact of migrating these specific containers between two compute nodes on a second set of key performance indicators regarding the two compute nodes; as well as The key performance indicators dashboard displays the first impact of adjusting the current number of containers corresponding to each of the multiple different microservices on the first set of key performance indicators and the second impact of migrating these specific containers between two compute nodes on the second set of key performance indicators.

14. A computer program product for automatically performing container scaling and migration for container-based microservices, the computer program product comprising a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by a computer to cause the computer to perform the following methods: The computer extracts a first set of features from each of the various microservices. The computer uses a trained prediction model and the first set of features extracted from each corresponding microservice to predict the number of containers required for each of the multiple different microservices at a future time point. The computer assigns scaling labels and scaling values ​​to each of the plurality of different microservices based on a predicted change in the number of containers required by each corresponding microservice at a future point in time, based on the current number of containers corresponding to each corresponding microservice. The computer automatically adjusts the current number of containers corresponding to each of the plurality of different microservices based on the scaling label and the scaling value assigned to each corresponding microservice; The computer extracts a second set of features from each of the multiple different microservices; as well as The computer determines the degree of microservice similarity among specific microservices in the plurality of different microservices based on the second set of features extracted from each corresponding microservice.

15. The computer program product according to claim 14, further comprising: The computer identifies a first group of containers running on a first group of computing nodes among a plurality of computing nodes, which have network latency values ​​higher than a network latency threshold level. as well as The computer determines a second set of compute nodes running a second set of containers among the plurality of compute nodes based on the determined degree of similarity between the microservices in the specific microservices. The second set of containers has a container similarity degree higher than the container similarity threshold level of the first set of containers running on the first set of compute nodes with a network latency value higher than the network latency threshold level.

16. The computer program product according to claim 15, further comprising: The computer identifies specific containers in the first group of containers and the second group of containers that share the same degree of container similarity. as well as The computer migrates specific containers that share the same degree of container similarity to the same computing node to reduce network latency.

17. The computer program product according to claim 16, further comprising: The computer uses causal inference to determine the first impact of adjusting the current number of containers corresponding to each of the plurality of different microservices on a first set of key performance indicators of these compute nodes with respect to the adjusted number of containers among the plurality of compute nodes; The computer uses the causal inference adjustment to determine the second impact of migrating these specific containers between two computing nodes on a second set of key performance indicators regarding the two computing nodes. as well as The computer displays, within a key performance indicator dashboard, the first impact of adjusting the current number of containers corresponding to each of the multiple different microservices on the first set of key performance indicators and the second impact of migrating these specific containers between two compute nodes on the second set of key performance indicators.

Images