Container pool management

Abstract

The invention relates to container pool management. A computer-implemented method for managing jobs using job processing pools. A computer system receives a job having a certain job type. The computer system identifies a job processing pool of the job processing pools for running jobs of the job type, wherein the job processing pool comprises job processors for running jobs of the job type, and wherein another job processing pool of the job processing pools contains other job processors for running jobs of a different job type. The computer system runs the job having the job type with the job processors of the job type in the job processing pool. According to other exemplary embodiments, a computer system and a computer program product for managing jobs are provided. Thus, exemplary embodiments can provide the technical effect of improving performance of running jobs in a computer system, wherein latency of running jobs can be reduced.

Description

Technical Field

[0001] This invention generally relates to improved computer systems, and more specifically to managing workloads in container pools. Background Technology

[0002] Machine learning models are trained by running model training jobs. These model training jobs can be submitted to a training service. The training service can accept model training jobs that run frequently within short time intervals. These jobs can complete the training of machine learning models within a short period of time. There is a requirement for short or immediate availability of resources to process these training jobs.

[0003] Currently, each model training job waits for resources in the cluster to become available to run it. The cluster could be, for example, a Kubernetes cluster including nodes running containerized applications. These containerized applications are designed to run model training jobs. When resources become available, the execution of the model training job is scheduled. Scheduling involves the time overhead of creating and initializing the containers required to run the model training job. Furthermore, the scheduling concept is to handle resources as fixed, with computation focused on selecting and assigning jobs to those resources.

[0004] This type of process may not reduce the waiting time for running model training jobs as expected, because the number of jobs increases and they change frequently.

[0005] Therefore, a method and apparatus that takes into account at least some of the problems discussed above, as well as other possible problems, is desired. For example, a method and apparatus that can overcome technical problems by improving the efficiency of running tasks such as model training jobs is desired. Summary of the Invention

[0006] According to an exemplary embodiment, a computer-implemented method is provided for managing jobs using job processing pools. A computer system receives jobs of a certain job type. The computer system identifies job processing pools within the job processing pools for running jobs of that job type, wherein each job processing pool includes a job processor for running jobs of that job type, and wherein another job processing pool within the job processing pool includes other job processors for running jobs of different job types. The computer system runs the job of that job type using the job processor of that job type in the corresponding job processing pool.

[0007] According to other exemplary embodiments, a computer system and computer program product for managing jobs are provided. Therefore, the exemplary embodiments can provide technical effects that improve the performance of running jobs in a computer system, including at least one of: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time.

[0008] When a computer system runs a job using a job processor of that job type in a job processing pool for that job type, an exemplary embodiment may allow the computer system to run the job using a job processor of that job type in that job processing pool in response to the availability of the job processor for that job type for running the job. Therefore, by using an available job processor having the same processing type as the job, the exemplary embodiment can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time. When a computer system runs a job using a job processor of that job type in a job processing pool for that job type, an exemplary embodiment may allow the computer system to determine whether a job processor of that job type can be added to the job processing pool to run a job of that job type; in response to determining that a job processor of that job type can be added to the job processing pool to run a job of that job type and that adding a job processor of that job type to the job processing pool does not exceed a set of resource constraints, the computer system adds a job processor of that job type to the job processing pool, and the computer system uses the job processor of that job type to run the job. Therefore, the exemplary embodiment can add job processors while taking into account a set of resource constraints, which can provide technical effects that improve the performance of running jobs in the computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time. When a computer system adds a job processor of a certain job type in response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type, an exemplary embodiment may remove a set of idle job processors to free up resources for adding the job processor, and then add a job processor of that job type to run jobs of that job type. Therefore, by freeing up resources, the exemplary embodiment can provide technical effects that improve the performance of running jobs in the computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time.When a computer system adds a job processor of a certain job type in response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type, an exemplary embodiment may allow the computer system to prioritize the job processing pool based on the total job priority of the job processing pool and the determination of the job processors in the job processing pool according to the new resource requirements of the job processing pool that need to be increased in size based on time-based job forecasts; remove idle job processors from the job processing pools with higher priority rankings so that sufficient resources are available for adding the job processors; and add the job processor of that job type to the job processing pool. Therefore, by removing idle jobs according to the ranking of the pools using the total priority, freeing up resources for adding new job processors, the exemplary embodiment can provide technical effects that improve the performance of running jobs in the computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. When a computer system adds a job processor of a certain job type in response to determining that a job processor of that job type can be added to the job processing pool to run a job of that job type, an exemplary embodiment may allow the computer system to identify a set of candidate jobs with lower priority than the job, wherein the resources freed up by transferring the set of candidate jobs are sufficient to add a job processor of that job type; the computer system places the set of candidate jobs in a waiting queue; and the computer system adds a job processor of that job type to the job processing pool. Therefore, by transferring lower-priority jobs to the waiting queue and freeing up resources for higher-priority jobs, the exemplary embodiment can provide the technical effect of improving the performance of running jobs in the computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. An exemplary embodiment may allow the computer system to remove idle job processors that have been idle for more than a threshold amount of time. Therefore, by selectively removing idle job processors, the exemplary embodiment can provide the technical effect of improving the performance of running jobs in the computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. An exemplary embodiment may allow a computer system to determine time-based job predictions for jobs and to coordinate the size of the job processing pool using these time-based job predictions.Therefore, by managing the job processing pool using predictions of future jobs, exemplary embodiments can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. When the computer system coordinates the size of the job processing pool using time-based job predictions, exemplary embodiments can allow the computer system to remove current job processors from the job processing pool that needs to be shrunk according to the time-based job predictions, and to add new job processors to the job processing pool based on a set of time-based job predictions and resource constraints. Therefore, by selectively adding and removing job processors from the job processing pool based on time-based job predictions, exemplary embodiments can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. When a computer system adds a new job processor to a job processing pool based on a set of time-based job predictions and resource constraints, an exemplary embodiment may allow the computer system to determine the additional resource requirements for the job processing pool that need to be increased in size according to the time-based job predictions; add a new job processor in response to determining that adding a new job processor to the job processing pool does not exceed the set of resource constraints; in response to determining that adding a new job processor to the job processing pool exceeds the set of resource constraints, the computer system sequentially sorts the job processing pool based on the total job priority of the job processing pool and the resource requirements of the job processors in the job processing pool according to the time-based job predictions for the jobs; and add a new job processor to the job processing pool in the order provided that the set of resource constraints does not exceed the set of resource constraints, wherein the set of resource constraints is selected from one of global resource constraints, consumer resource constraints, or per-container-type constraints. Therefore, by considering resource constraints when adding job processors based on the time predictions of jobs, exemplary embodiments may provide technical effects that improve the performance of running jobs in a computer system, including at least one of: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by job processors at any given time. An exemplary embodiment may also allow a computer system to transfer a current job from the job processor in the job processing pool to a dedicated job processor outside the job processing pool in response to the current job in the job processor running for a period of time exceeding a running time threshold.Therefore, by running jobs whose execution time exceeds a certain desired threshold for running jobs in the job processing pool in a separate job processor for that job, exemplary embodiments can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a cloud computing environment in which exemplary embodiments can be implemented;

[0010] Figure 2 This is a schematic diagram illustrating an abstract model layer according to an exemplary embodiment;

[0011] Figure 3 This is a diagram of a data processing system network in which illustrative embodiments can be implemented;

[0012] Figure 4 This is a block diagram of a job processing environment according to an exemplary embodiment;

[0013] Figure 5 This is a schematic diagram of a container pool of jobs with different job types according to an exemplary embodiment;

[0014] Figure 6 This is a description of resource limitations according to an exemplary embodiment;

[0015] Figure 7 This is a flowchart of the process of managing jobs according to an exemplary embodiment;

[0016] Figure 8 This is a flowchart illustrating the process of running a job according to an exemplary embodiment;

[0017] Figure 9 This is a flowchart illustrating the process of running a job according to an exemplary embodiment;

[0018] Figure 10 This is a flowchart illustrating the process of adding a job processor according to an exemplary embodiment;

[0019] Figure 11 This is a flowchart illustrating the process of adding a job processor according to an exemplary embodiment;

[0020] Figure 12 This is a flowchart illustrating the process of adding a job processor according to an exemplary embodiment;

[0021] Figure 13 This is a flowchart illustrating the process of managing processing pool resources according to an example embodiment;

[0022] Figure 14 This is a flowchart illustrating the process of managing jobs using time-based job prediction management according to an example embodiment;

[0023] Figure 15 This is a flowchart illustrating the process of coordinating a job processing pool according to an exemplary embodiment;

[0024] Figure 16 This is a flowchart illustrating the process of adding a new job processor according to an exemplary embodiment;

[0025] Figure 17 This is a flowchart of the process of managing jobs according to an exemplary embodiment;

[0026] Figure 18 This is a flowchart illustrating the process of performing time-based job prediction according to an exemplary embodiment;

[0027] Figure 19 This is a flowchart illustrating the process of coordinating a container pool according to an exemplary embodiment;

[0028] Figure 20 This is a flowchart illustrating the process of removing an idle container according to an exemplary embodiment;

[0029] Figure 21 This is a flowchart illustrating the process of job preemption according to an exemplary embodiment;

[0030] Figure 22 This is a block diagram of a data processing system according to an exemplary embodiment. Detailed Implementation

[0031] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0032] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals emitted through wires.

[0033] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0034] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of this invention.

[0035] This document describes various aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0036] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in the blocks of the flowchart and / or block diagram.

[0037] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0038] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0039] It should be understood that while this disclosure includes a detailed description of cloud computing, the implementation of the teachings given herein is not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0040] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0041] The features are as follows:

[0042] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0043] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0044] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0045] Rapid flexibility: The ability to provide capacity quickly and flexibly, automatically scaling down and up rapidly in some situations to scale up rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0046] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.

[0047] The service model is as follows:

[0048] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0049] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created by the consumer or acquired using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.

[0050] Infrastructure as a Service (IaaS): The capabilities offered to consumers are processing, storage, networking, and other basic computing resources that enable consumers to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0051] The deployment model is as follows:

[0052] Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0053] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0054] Public cloud: Makes cloud infrastructure available to the public or large industry groups and is owned by an organization that sells cloud services.

[0055] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0056] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0057] See now Figure 1 This illustration shows a schematic diagram of a cloud computing environment in which an exemplary embodiment may be implemented. In this example, the cloud computing environment 100 includes one or more cloud computing nodes 110 to which local computing devices used by cloud consumers can communicate, such as, for example, personal digital assistants (PDAs) or smartphones 120A, desktop computers 120B, laptop computers 120C, and / or automotive computer systems 120N.

[0058] Cloud computing nodes 110 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 100 to provide infrastructure, platforms, and / or software as services to cloud consumers who do not need to maintain resources on their local computing devices. It should be understood that... Figure 1 The types of computing devices 120A-120N shown are intended to be illustrative only, and computing node 110 and cloud computing environment 100 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0059] See now Figure 2 This illustrates a schematic diagram of an abstract model layer according to an exemplary embodiment. The set of functional abstract layers shown in this example can be composed of, for example... Figure 1 The cloud computing environment provided is 100% cloud computing environment. This should be understood in advance. Figure 2 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As shown, the following layers and corresponding functions are provided.

[0060] The abstraction layer 200 of the cloud computing environment includes a hardware and software layer 202, a virtualization layer 204, a management layer 206, and a workload layer 208. The hardware and software layer 60 includes the hardware and software components of the cloud computing environment. Hardware components may include, for example: a mainframe 210; a RISC (Reduced Instruction Set Computer) based server 212; a server 214; a blade server 216; a storage device 218; and network and networking components 220. In some embodiments, software components include, for example, network application server software 222 and database software 224.

[0061] The virtualization layer 204 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 226; virtual storage 228; virtual network 230, including virtual private network; virtual application and operating system 232; and virtual client 234.

[0062] In one example, management layer 206 provides the functionality described below. Resource provisioning 236 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 238 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 240 provides access to the cloud computing environment for consumers and system administrators. Service level management 242 provides cloud resource allocation and management to ensure that required service levels are met. Service level agreement (SLA) planning and fulfillment 244 provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.

[0063] Workload layer 208 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that may be provided by workload layer 208 include: mapping and navigation 246; software development and lifecycle management 248; virtual classroom instruction delivery 250; data analytics and processing 252; transaction processing 254; and container orchestration 256.

[0064] In this example, container orchestration 256 provides a means of management. Figure 1 Applications or access in cloud computing environments 100 Figure 1 The service is deployed in the network at the physical location of the cloud computing environment 100. In the illustrative example, container orchestration 256 can provide automated application deployment scaling and management. For example, container orchestration 256 can automate the deployment, scaling, and operation of containers across different host clusters. In this illustrative example, a container image is an executable software package that includes all the components required to run an application. A container is a runnable instance of an image.

[0065] These containers can be organized into pools of 256 containers to run jobs. For example, a container orchestration of 256 can serve as a training service operation, where training jobs are used to train learning models.

[0066] See now Figure 3 The illustration shows a data processing system network in which illustrative embodiments can be implemented. Network data processing system 300 is a computer network in which illustrative embodiments can be implemented. Network data processing system 300 includes network 302, which is a medium for providing communication links between different devices and computers connected together within network data processing system 300. Network 302 may include connections such as wired, wireless communication links, or fiber optic cables.

[0067] In the illustrated example, server computers 304 and 306 are connected to network 302 along with storage unit 308. Additionally, client device 310 is connected to network 302. As shown, client device 310 includes client computers 312, 314, and 316. Client device 310 can be, for example, a computer, workstation, or network computer. In the illustrated example, server computer 304 provides information such as boot files, operating system images, and applications to client device 310. Furthermore, client device 310 may also include other types of client devices, such as mobile phone 318, tablet computer 320, and smart glasses 322. In this illustrative example, server computer 304, server computer 306, storage unit 308, and client device 310 are network devices connected to network 302, where network 302 is the communication medium for these network devices. Some or all of client devices 310 can form an Internet of Things (IoT), where these physical devices can connect to network 302 and exchange information with each other through network 302.

[0068] In this example, client device 310 is a client of server computer 304. Network data processing system 300 may include additional server computers, client computers, and other devices not shown. Client device 310 is connected to network 302 using at least one of a wired connection, a fiber optic connection, or a wireless connection.

[0069] The program code located in the network data processing system 300 can be stored on a computer-recordable storage medium and downloaded to the data processing system or other devices for use. For example, the program code can be stored on a computer-recordable storage medium on a server computer 304 and downloaded to a client device 310 via network 302 for use on the client device 310.

[0070] In the example shown, network data processing system 300 is the Internet, where network 302 represents a collection of global networks and gateways that communicate with each other using the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol suite. The core of the Internet is a backbone of high-speed data communication lines between master nodes or host computers, consisting of thousands of commercial, government, educational, and other computer systems routing data and messages. Of course, network data processing system 300 can also be implemented using many different types of networks. For example, network 302 can include at least one of the following: the Internet, intranet, local area network (LAN), metropolitan area network (MAN), or wide area network (WAN). Figure 3 This is intended as an example, not as a limitation on the architecture of different illustrative embodiments.

[0071] As used in this article, "multiple" when referring to projects means one or more projects. For example, "multiple different types of networks" means one or more different types of networks.

[0072] Furthermore, the phrase "at least one" when used with a list of items means that different combinations of one or more of the listed items can be used, and only one item from each of the items in the list may be required. In other words, "at least one" means that any combination and any number of items in the list can be used, but not all items in the list are required. Items can be specific objects, things, or categories.

[0073] For example, but not limited to, "at least one of project A, project B, or project C" can include project A, project A and project B, or project B. The example could also include project A, project B and project C, or project B and project C. Of course, any combination of these projects can exist. In some illustrative examples, "at least one" can be, for example, but not limited to, two projects A; one project B; and ten projects C; four projects B and seven projects C; or other suitable combinations.

[0074] The container orchestration platform 330 can be, for example, Kubernetes. ® Architecture, environment, etc. However, it should be understood that the description of the illustrative example using Kubernetes is only intended to represent an example architecture and not a limitation as an illustrative embodiment. The container orchestration platform 330 may also be referred to as a container orchestration system.

[0075] The container orchestration platform 330 provides a platform, such as a job manager 332, for automating the deployment, scaling, and operation of customer applications. The job manager 332 runs jobs 334 received from one or more client devices 310. Jobs 334 can be received dynamically. Furthermore, it is possible to receive jobs 334 of different job types at different frequencies.

[0076] The container orchestration platform 330 provides automated deployment, scaling, and operation of container pools 336 for running jobs 334. Each container pool in the container pools 336 includes multiple containers 338 for running jobs 334 received by the job manager 332. Containers 338 in the container pools are used for processing within jobs 334. In other words, containers 338 in the container pools are used to process specific types of jobs.

[0077] A container is a runnable instance of a container image. A container image contains a package of executable software components required to run an application. A container is a standard unit of software for an application, packaging program instructions and all its dependencies so that the application can run on multiple computing environments. Containers isolate software from the environment in which they run and ensure that containers function uniformly across different environments. An application's container can share the operating system kernel on the machine with other containers used for other applications. Therefore, not every container running on a machine requires an operating system.

[0078] In this illustrative example, job manager 332 sends job 334 to different container pools to run job 334 based on its job type. By sending job 334 to a specific container pool based on its job type, job manager 332 improves the efficiency of running jobs. When containers running jobs are designed or configured to run jobs with that job type more efficiently than running another job with a different job type, organizing containers 338 into container pools 336 based on job type increases throughput in job processing and reduces the waiting time for resources to become available while running job 334.

[0079] In this illustrative example, the job manager 332 can adjust the size of the container pool 336 based on the type of job 334 that has been received or is predicted to be received for processing. Furthermore, predictive analytics can be used to predict when job 334 will be received over time. This prediction can be used to adjust the container pool 336 to improve the efficiency of running job 334 by the container orchestration platform 330.

[0080] Therefore, the container orchestration platform 330 can improve the performance of jobs running within the network data processing system 300, including at least one of the following: increasing the total throughput of job processing, reducing the waiting time for running jobs, or reducing the resources held by the job processor at any given time.

[0081] See now Figure 4 This illustrates a block diagram of a job processing environment according to an illustrative embodiment. In this illustrative example, the job processing environment 400 includes components that can, for example... Figure 1 The hardware shown in the cloud computing environment 100 and Figure 3The components implemented in the network data processing system 300.

[0082] As shown in the figure, the job processing environment 400 is an environment in which job 402 is processed by a job processing pool 404 having a job processor 406.

[0083] Job 402 can take many different forms. For example, job 402 can be selected from at least one of a model training job for training machine learning models, a scheduling job for semiconductor manufacturing operations, an order processing job, or other suitable job or operation type.

[0084] The job processor 406 can take many different forms. For example, the job processor 406 can be selected from at least one of containers, threads, processes, applications, operating system instances, virtual machines, hosts, clusters, processing units, or other suitable types of processing components.

[0085] In this example, the job management system 412 can manage the job processing pool 404 and run jobs 402. In this example, the job management system 412 includes a computer system 414 and a job manager 420.

[0086] The job manager 420 resides in the computer system 414 and can be implemented using software, hardware, firmware, or a combination thereof. When using software, the operations performed by the job manager 420 can be implemented in program instructions configured to run on hardware such as a processor unit. When using firmware, the operations performed by the job manager 420 can be implemented in program instructions and data, and stored in persistent memory for execution on the processor unit. When using hardware, the hardware may include circuitry that operates to perform the operations in the job manager 420.

[0087] In the illustrative examples, the hardware may take the form of at least one of the following: a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform multiple operations. Using a programmable logic device, the device can be configured to perform multiple operations. The device can be reconfigured later or can be permanently configured to perform multiple operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field-programmable logic arrays, field-programmable gate arrays, and other suitable hardware devices. Furthermore, these processes can be implemented in organic components integrated with inorganic components and can consist entirely of organic components other than human-made ones. For example, these processes can be implemented as circuits in organic semiconductors.

[0088] Computer system 414 is a physical hardware system that includes one or more data processing systems. When multiple data processing systems exist in computer system 414, these data processing systems communicate with each other using a communication medium. The communication medium can be a network. The data processing systems can be selected from at least one of a computer, a server computer, a tablet computer, or some other suitable data processing system.

[0089] As shown in the figure, computer system 414 includes multiple processor units 422 capable of executing program instructions 424 to implement the process in the example. As used herein, the processor units among the multiple processor units 422 are hardware devices and consist of hardware circuitry, such as hardware circuitry on an integrated circuit that responds to and processes instructions and program code that operate the computer. When the multiple processor units 422 execute the program instructions 424 of the process, the multiple processor units 422 are one or more processor units that can be on the same computer or different computers. In other words, the process can be distributed among processor units on the same or different computers in the computer system. Furthermore, the multiple processor units 422 can be processor units of the same type or different types. For example, multiple processor units can be selected from at least one of a single-core processor, a dual-core processor, a multi-processor core, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or other types of processor units.

[0090] In this example, the job manager 420 in the job management system 412 manages running jobs 402 using job processing pools 404 based on job type 426. Each job processing pool is configured to handle jobs 402 of the same job type. For example, job processing pool 428 is used to run jobs 402 of job type 430 in job type 426. The job processors 406 in job processing pool 428 of job processing pool 404 are configured or designed to run jobs 402, all of which are job type 430. In this example, another job processing pool in job processing pool 404 may include other job processors 406 for running jobs 402 with a different job type than job type 430 of job processing pool 404. Therefore, different job processing pools can handle jobs of different job types in job type 426.

[0091] In this example, job manager 420 can receive job 432 with a job type 430. Job manager 420 can identify job processing pool 428 in job processing pool 404 for running job 402 with job type 430. Job processing pool 428 includes job processor 406 for running job 402 with job type 430. Job manager 420 uses job processor 434 in job processing pool 428 with job type 430 to run job 432 with job type 430.

[0092] When running job 432, job manager 420 may use job processor 434 in job processing pool 428 for job type 430 to run job 432 in response to the availability of job processor 434 for job type 430.

[0093] In another example, when a job processor is unavailable, the job manager 420 can perform several different steps to run job 432. For example, the job manager 420 can determine whether a job processor 434 of job type 430 can be added to the job processing pool 428 to run job 432 of job type 430. The job manager 420 can add the job processor 434 of job type 430 to the job processing pool 428 in response to the possibility that the job processor 434 of job type 430 can be added to the job processing pool 428 to run job 432 of job type 430, and that adding the processor 434 of job type 430 to the job processing pool 428 does not exceed the set of resource limits 436. The job manager 420 can then use the job processor 434 to run job 432.

[0094] When adding job processor 434, job manager 420 can remove a set of idle job processors 444 to free up resources used for adding job processor 434. In the example, a job processor may be in a waiting-for-job-idle state or running a job. In both states, the job processor has processing resources. Job manager 420 can add job processor 434 of job type 430 in response to freeing up resources to run job 432 of job type 430.

[0095] In another example, when adding a job processor 434, the job manager 420 can sort the job processing pool 404 in order 446 based on the total job priority 448 of the job processing pool 404 and the resource requirements 450 for the job processors 406 in the job processing pool 404. In this example, jobs 402 can have different priorities in addition to having different job types.

[0096] A total job priority can be determined for each job processing pool. The total job priority of a job processing pool is the total priority or a subset of the priorities of the jobs currently being processed in the job processing pool.

[0097] Job manager 420 can remove idle job processor 444 from job processing pool 404 with a higher ranking 446, so that sufficient resources are available for adding job processor 434. In response to removing idle job processor 444, job manager 420 can add job processor 434 of job type 430 to job processing pool 428 to free up sufficient resources.

[0098] In another example, when adding a job processor 434, the job manager 420 can identify a group of candidate jobs 452 with a priority 454 lower than job 402. The resources freed up by transferring this group of candidate jobs 452 are sufficient to add a job processor 434 for job type 430. In this example, the job manager 420 places the group of candidate jobs 452 in the waiting queue 456 and places the job processor 434 for job type 430 in the job processing pool 428.

[0099] When managing resources, the job manager 420 can also selectively remove job processors 406. For example, the job manager 420 can remove a job processor 406 that has run for more than a threshold amount of time 458. In another example, the job manager 420 can, in response to the current job 460's running time exceeding a running time threshold 464, transfer the current job from job processor 434 in the job processing pool 428 to a dedicated job processor 462 outside the job processing pool 404.

[0100] In this example, the value of the runtime threshold 464 can be based on analysis of past jobs. The selection of the runtime threshold 464 allows jobs with longer than expected or anticipated runtimes to be run in job processors outside of job processing pool 404, so that job processors 406 in job processing pool 604 do not maintain a longer or anticipated duration than the processing of job 402.

[0101] Conversely, dedicated job processor 462 can be used to handle these longer-running jobs. Job runtimes may be longer than expected or anticipated due to factors such as input, latency, dependencies, interactions, or other factors. Therefore, the job management system 412 can be optimized to handle shorter-running jobs with runtimes below the runtime threshold 464. Long-running jobs can be handled using job processors outside of the job processing pool 404.

[0102] Furthermore, the job manager 420 can implement time-based job prediction 466 to predict the jobs 402 that will be submitted over time. In this example, the job manager 420 can use time-based job prediction 466 to manage the number of job processors 406 and the number of job processing pools 404 over different time periods.

[0103] In this example, time-based job prediction 466 uses historical job processing data for different job types. Time-based job prediction 466 can be achieved by building a mathematical model that captures trends in historical job data. This model can then be used to predict future jobs. In other examples, historical job data can be used to train a machine learning model to perform time-based job prediction 466.

[0104] In one example, job manager 420 can determine a time-based job forecast 466 for job 402. Job manager 420 can use the time-based job forecast 466 for job 402 to coordinate the size of job processing pool 404. This coordination can be performed by adding or removing job processors in one or more job processing pools 404.

[0105] When coordinating job processing pool 404, job manager 420 may remove the current job processor 468 from job processing pool 404 that needs to be reduced in size according to time-based job forecast 466. Job manager 420 may add a new job processor 470 to job processing pool 404 based on a set of time-based job forecast 466 and resource constraints 436. In other words, a new job processor 470 may be added based on time-based job forecast 466, wherein the addition of a new job processor 470 does not exceed the set of resource constraints 436.

[0106] When adding a new job processor 470, the job manager 420 can determine a new resource requirement 472 for the job processing pool 404 that needs to be increased in size based on time-based job forecasts 466. In response to determining that adding the new job processor 470 to the job processing pool 404 does not exceed the set of resource limits 436, the job manager 420 can add the new job processor 470. In response to determining that adding the new job processor 470 to the job processing pool 404 exceeds the set of resource limits 436, the job manager 420 can sort the job processing pool 404 in order 446 based on the total job priority 448 of the job processing pool 404 and the resource requirement 450 for the job processors 406 in the job processing pool 404 based on the time-based job forecasts 466 for job 402. The job manager 420 can add the new job processor 470 to the job processing pool 404 in order 446, without exceeding the set of resource limits 436.

[0107] In one example, one or more technical solutions are provided to address the technical problem of running jobs efficiently at a desired level. Therefore, one or more technical solutions can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time. In this example, the total throughput of running jobs can be increased over different time intervals. Furthermore, the waiting time for different types of jobs can be reduced. In other words, the job processor can be made available immediately or as quickly as possible for running jobs. Additionally, the management of container resources held between different containers can be reduced at any given time. Moreover, these efficiencies can be taken into account when jobs are running.

[0108] Computer system 414 can be configured to use software, hardware, firmware, or a combination thereof to perform at least one of the steps, operations, or actions described in the various examples. Therefore, computer system 414 operates as a dedicated computer system, wherein the job manager 420 in computer system 414 enables improved efficiency in running jobs on computer system 415. Specifically, the job manager 420 transforms computer system 414 into a dedicated computer system compared to currently available general-purpose computer systems without a job manager 420.

[0109] In the example, the use of job manager 420 in computer system 414 integrates processes into the actual application of running jobs to improve the performance of computer system 414. In other words, job manager 420 in computer system 414 targets the actual application of processes integrated into computer system 414, which identifies the job processing pool based on the job type of the job to be run.

[0110] Figure 4 The description of the job processing environment 400 does not imply a physical or architectural limitation on how the illustrative embodiments are implemented. Other components besides those illustrated, or those replacing the illustrated components, may be used. Some components may be unnecessary. Furthermore, boxes are provided to illustrate some functional components. When implemented in exemplary embodiments, one or more of these boxes may be combined, divided, or combined and divided into different boxes.

[0111] For example, job processing pool 404 can have two job processing pools that process jobs 402 of the same job type. In this example, job processing pool 404 still covers the processing of all job types 426 of job 402. In some examples, duplicate job processing pools may appear.

[0112] Next, proceed to Figure 5 This illustrates a container pool of jobs with different job types according to an exemplary embodiment. In this example, job 500, job 502, and job 504 are... Figure 4 Example of assignment 402. These assignments have different assignment types. This example presents three distinct assignment types. As shown in the figure, assignment 500 has assignment type 1, assignment 502 has assignment type 2, and assignment 504 has assignment type 3.

[0113] These jobs can be run by containers, such as container 506 in container pool 508, container 510 in container pool 512, and container 514 in container pool 516. These containers are examples of the implementation of job processor 406, and the container pools are... Figure 4 Example of the implementation of the 404 job processing pool.

[0114] Each container in the container pool is uniquely used to run a specific type of job. For example, container 506 is used to run type 1 jobs, container 510 is used to run type 2 jobs, and container 514 is used to run type 3 jobs. In this example, a container of a specific type can run a single job of the same type at any given time.

[0115] Container 506 of job type 1 can run job 500 of job type 1. Container 510 of job type 2 can run job 502 of job category 2. Container 514 of job type 3 can run job 504 of job category 3.

[0116] As an example implementation, this provides an explanation of the container types for running different types of jobs. This explanation is not intended to limit the possible implementations of other examples. For example, in other examples, a pool could contain groups of container pods. In other examples, other types of job processors could be used, such as threads, processes, job processor virtual machines, or other suitable types of job processors. Although this example describes three container pools, the number of container pools used depends on the number of job types to be run. In this example, there is a one-to-one relationship between the container types associated with the job types. For example, if there are five types of jobs, there could be five container pools with containers dedicated to running specific job types.

[0117] Next, proceed to Figure 6 This illustrates an example of resource limits according to an exemplary embodiment. In this example, resource limit 600 is... Figure 4 An example of the set of resource limits 436. As shown in the figure, resource limit 60 is selected from at least one of global resource limit 602, job processor resource limit 604, and consumer resource limit 606.

[0118] Global resource limit 602 can be a resource consumption limit for all job processors across all job processing pools. Job processor resource limit 604 can be a resource limit on each job processor. Different job processors can use different amounts of resources depending on the job type. Consumer resource limit 606 is a consumption limit that can be set for the consumer submitting the job or by the consumer. Any one or a combination of these resource limits can be considered when adding job processors.

[0119] In other examples, other types of resource limits besides resource limit 600 can be used. For example, there could be pool resource limits that are set based on each job processing pool.

[0120] Next, proceed to Figure 7 This illustrates a flowchart of a process for managing jobs according to an exemplary embodiment. Figure 7 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, Figure 7 The process can be done in Figure 4 It is implemented in the job manager 420 of the computer system 416.

[0121] The process first receives a job with a certain job type (step 700). The process identifies a job processing pool in the job processing pool for running jobs of that job type (step 702). In step 702, the job processing pool includes a job processor for running jobs of that job type, wherein another job processing pool in the job processing pool contains other job processors for running jobs of different job types.

[0122] The process uses the job processor of that job type in the job processing pool for that job type to run the job with that job type (step 704). The process then terminates.

[0123] Now for reference Figure 8 This flowchart illustrates the process of running a job according to an exemplary embodiment. Figure 7 The implementation of step 704.

[0124] The process responds to the availability of the job processor in the job processing pool for that job type to run the job, and then runs the job with that job type using the job processor in the job processing pool for that job type (step 800). The process then terminates.

[0125] refer to Figure 9This illustrates a flowchart of the process of running a job according to an exemplary embodiment. Figure 9 The process shown is Figure 7 Example of step 704 in the example.

[0126] The process first determines whether a job processor of the job type can be added to the job processing pool to run jobs of that job type (step 900). In response to determining that a job processor of the job type can be added to the job processing pool to run jobs of that job type and determining that adding a job processor of the job type to the job processing pool does not exceed the set of resource limits, the process adds the job processor of the job type to the job processing pool (step 902).

[0127] The process runs the job using the job processor for that job type (step 904). The process then terminates.

[0128] refer to Figure 10 This illustrates a flowchart of the process of adding a job processor according to an exemplary embodiment. Figure 10 The process shown is Figure 9 Example of step 902 in the example.

[0129] The process first removes a set of idle job processors to free up resources for adding the job processor (step 1000). In this example, step 1000 is an optional step. The process adds a job processor for that job type to run a job with that job type (step 1002). The process then terminates.

[0130] exist Figure 11 The diagram shows a flowchart of the process of adding a job processor according to an exemplary embodiment. Figure 11 The process shown is Figure 9 Example of step 902.

[0131] The process first sorts the job processing pools in order based on the total job priority and the resource requirements of the job processors in the pools (step 1100). The process removes idle job processors from the job processing pools with higher order rankings so that sufficient resources are available for adding job processors (step 1102).

[0132] The process adds the job processor for that job type to the job processing pool (step 1104). The process then terminates.

[0133] refer to Figure 12 This illustrates a flowchart of the process of adding a job processor according to an exemplary embodiment. Figure 12 The process shown is Figure 9 Example of step 902.

[0134] The process first identifies a group of candidate jobs with lower priority than the current job (step 1200). In step 1200, the resources freed up by transferring this group of candidate jobs are sufficient to add a job processor for this job type. The process then places this group of candidate jobs in a waiting queue (step 1202).

[0135] The process adds the job processor for that job type to the job processing pool (step 1204). The process then terminates.

[0136] refer to Figure 13 This illustrates a flowchart of a process for managing resources for a processing pool according to an exemplary embodiment. Figure 13 The process shown is in Figure 7 Examples of additional steps that can be used in the process.

[0137] The process first removes idle job processors that have been idle for more than a threshold amount of time (step 1300). In response to the current job's execution time exceeding the execution time threshold, the process transfers the current job from a job processor in the job pool to a dedicated job processor outside the job pool (step 1302). The process then terminates.

[0138] In step 1302, the runtime threshold can be selected based on various factors. For example, this threshold can be selected to improve the performance of the currently running job. In this example, the threshold could be based on an analysis of past jobs that have been run and in which other runtime thresholds were used.

[0139] In this example, a job that exceeds the runtime threshold is selected. The job is stopped and placed in a queue for execution using a dedicated container allocated and started for that job. This container resides outside the container pool. Information about the job exceeding the runtime threshold is logged. This logged information can be used to identify similar jobs. These types of jobs can then be moved to a dedicated job processor instead of running in the container pool. In other words, jobs that do not meet the characteristics required for running in a container pool can be handled using a different mechanism.

[0140] In another example, it is not necessary to stop jobs that exceed the runtime threshold. If the job that exceeds the runtime threshold has a higher priority, the job can be allowed to run, and the job processor can be removed from the processor pool.

[0141] refer to Figure 14 This illustrates a flowchart of a process for managing jobs using time-based job prediction management according to an exemplary embodiment. Figure 14 The process shown is possible Figure 7 Examples of additional steps used in the process.

[0142] The process first determines a time-based job forecast for the jobs (step 1400). The process uses the time-based job forecast to coordinate the size of the job processing pool (step 1402). The process then terminates.

[0143] Next, turn to Figure 15 The diagram illustrates a flowchart of the process of coordinating a job processing pool according to an exemplary embodiment. Figure 15 The process shown is Figure 14 An example of the implementation of step 1402.

[0144] The process first removes the current job processor from the job processing pool that needs to be reduced in size based on time-based job forecasts (step 1500). The process then adds a new job processor to the job processing pool based on a set of time-based job forecasts and resource constraints (step 1502). The process then terminates.

[0145] Now go to Figure 16 This illustrates a flowchart of the process of adding a new job processor according to an exemplary embodiment. Figure 16 The process shown is Figure 15 An example of the implementation of step 1502.

[0146] The process first determines the additional resource requirements for the job processing pool that need to be increased in size based on time-based job forecasts (step 1600). In response to determining that adding new job processors to the job processing pool does not exceed the set of resource limits, the process adds new job processors, but not exceeding the set of resource limits (step 1602).

[0147] In response to determining that adding a new job processor to the job processing pool exceeds the set of resource limits, the process prioritizes the job processing pool based on the overall job priority of the job processing pool and the resource requirements of the job processors in the job processing pool according to time-based job forecasts for the jobs (step 1604). The process adds new job processors to the job processing pool in order, but not exceeding the set of resource limits (step 1606). The process then terminates. In step 1606, the set of resource limits is selected from a global resource limit, a consumer resource limit, or a per-container type limit.

[0148] Next, turn to Figure 17 This illustrates a flowchart of the process of managing jobs according to an exemplary embodiment. Figure 17 The process can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in... Figure 4It is implemented in the job manager 420 of the computer system 416.

[0149] The process first determines a time-based job forecast for incoming jobs (step 1700). The process then coordinates the container pool based on the time-based job forecast (step 1702). In this example, steps 1700 and 1702 are optional steps.

[0150] The process receives a job to be processed (step 1704). In step 1704, the job has a certain job type. The process determines whether there is a container of that job type available to run the job (step 1706). In step 1706, it can be determined whether there is an available container in the pool for that job type by tracking the containers currently being used by the job in the container pool at any given time and comparing the number of currently used containers with the total number of containers in the pool. If a container is available, the process runs the job in that container (step 1708), and the process then terminates.

[0151] Referring again to step 1706, if the container is unavailable, determine whether a container of this job type can be added based on resource limitations (step 1710). If a container of this job type cannot be added (e.g., due to exceeding resource limits), then, if job preemption is supported, the process attempts to perform job preemption; otherwise, if job preemption is not supported, the process places the job in a waiting queue (step 1712). The job preemption process is as follows: Figure 21 As shown. The process then terminates.

[0152] Referring again to step 1710, if a container of this job type can be added, then if additional resources are needed, the process removes the idle container to free up resources for the new container (step 1714).

[0153] The process adds the container for that job type to the pool of containers for that job type (step 1716). Then, the process proceeds to step 1708 as described above.

[0154] Furthermore, the process waits for a job completion event or timer expiration from step 1702 (step 1718). Upon receiving a job completion event or timer expiration, the process removes idle containers to release resources (step 1720). The target for removing idle containers can be specified, for example, based on the amount of requested resources that should be released, the number of requested idle containers that should be removed, or an idle-related threshold for the containers to be removed (e.g., idle time exceeding a certain time threshold). The process identifies jobs that have exceeded runtime thresholds and switches these jobs to dedicated containers outside the container pool (step 1722). The process then terminates.

[0155] Now for reference Figure 18 This illustrates a flowchart of the process of performing time-based job prediction according to an exemplary embodiment. Figure 18 The process is Figure 17 An example of an implementation of step 1700.

[0156] The process first identifies historical information about prior jobs (step 1800). In step 1800, the historical information for each job includes attributes such as: (1) the calendar time when the job entered; (2) the type of job and the associated type of the container running the job; (3) the priority of the job; and (4) processing metrics—such as waiting time, runtime, and resources consumed. The process uses the historical information about prior jobs to determine a function for predicting future jobs (step 1802).

[0157] The process receives the time period for which job prediction needs to be performed (step 1804). The process uses the function to predict the jobs within that time period (step 1806). The process then terminates.

[0158] In this example, the function's output could be a predicted number of jobs for a given time period. The predicted jobs correspond to tuples of time intervals within that period. Each tuple can include the required container type and the required number of containers. These tuples can be aggregated or combined to determine the container types and the total number of containers of that type required for that time period.

[0159] Transfer to Figure 19 The diagram illustrates a flowchart of the process of coordinating a container pool according to an exemplary embodiment. Figure 19 The process is Figure 17 An example of an implementation of step 1702 in [the document]. In this example, time-based job forecasting can be used to reconcile the current size of the pool to the predicted size for the upcoming time period. This reconciliation can take into account various resource constraints. The process can add containers to pools with forecast values ​​higher than their current size, and remove containers from pools with forecast values ​​lower than their current size. In this example, if there is no container pool for a job type in the forecast for that time period, the container pool can also be emptied and removed. If the predicted pool size is the same as the current pool size, the container pool size is not changed. Furthermore, if resource constraints are met, the process can use a benefit-cost metric calculated for the pool to select the container pool to which to add containers.

[0160] The process first receives a forecast for a given time period (step 1900). The process then determines the number and type of containers required for that time period (step 1902). For example, the forecast might indicate that 10 containers of job type A, 5 containers of job type B, and 17 containers of job type C are needed for a specific time period.

[0161] The process determines the difference for each container pool (step 1904). In step 1904, the difference can be determined by subtracting the current size of each container pool from its predicted size. The process removes containers from the pools that need to be shrunk (step 1906).

[0162] The process then determines the additional resource requirements for the container pool that needs to be enlarged (step 1908). In step 1908, the additional resources required to run the required containers are determined.

[0163] For each container pool where the number of containers to be added exceeds the pool-based limit, the process reduces the number of containers to be added to meet the limit (step 1910). In step 1910, separate limits can be set for different container pools to restrict containers for specific job types. These limits are based on the container pool limit.

[0164] The process determines whether adding the required container to the container pool exceeds the global resource limit or the consumer resource limit (step 1912). If it is determined that adding the required container to the container pool will not exceed the global resource limit and the consumer resource limit, the process adds the required container to the container pool (step 1914). The process then terminates.

[0165] Referring again to step 1912, if adding required containers to a container pool exceeds global or consumer resource limits, the process determines the ratio of total job priority to resource requirements on a container pool-by-container basis (step 1916). In step 1916, the job priorities assigned to jobs for each container pool over a time period are aggregated to provide a total (e.g., average, median, or average) job priority for each container pool. The total job priority for each container pool is divided by the resource requirements of containers of that container type in that container pool. In this example, a higher ratio indicates a better benefit-to-cost ratio compared to a lower ratio.

[0166] The process sorts the container pools according to the ratios determined for each pool to produce an order (step 1918). In step 1918, the container pools can be ranked in descending order from the highest ratio to the lowest ratio. The higher-ranked container pools are better candidates for adding containers in terms of the benefits and costs of adding containers.

[0167] The process uses this sequence to add containers to the container pool, without exceeding the consumer resource limit, until the global resource limit is reached (step 1920). The process then terminates.

[0168] Next, proceed to Figure 20 This illustrates a flowchart of the process of removing an idle container according to an exemplary embodiment. The example is... Figure 17 Step 1714 in the process can be used as one way to remove idle containers. The process first identifies container pools with idle containers (step 2000). In step 2000, an idle container can be a container that has not run a job for more than a threshold time. If time-based job forecasting is available, a container pool in which the number of idle containers exceeds the forecasted required number of containers, or exceeds the forecasted minimum required number of containers, can be identified as a container pool with idle containers. For a given future time interval, the forecasting function provides the number of each container type required for that time interval. The function can also provide a range for each container type, where the range is the minimum required number and the maximum required number, or the function can provide any combination of the minimum, maximum, and required numbers for each container type.

[0169] The process determines the maximum number of containers that can be removed from each container pool (step 2002). In step 2002, the maximum number of containers is the number of idle containers in the container pool when the time-based job forecast is unavailable. If the time-based job forecast is available, the number of idle containers that can be removed refers to the number of idle containers in the container pool that exceed the forecast's required number of containers or exceed the forecast's minimum required number of containers.

[0170] The process sorts the container pools sequentially based on the total job priority and the resource requirements of the containers within the pool (step 2004). In step 2004, a job priority-to-size ratio is generated for each container pool. These ratios are then sorted in ascending order. As a result, because lower job priorities correspond to higher resource requirements and lower cost-benefit ratios, a container pool with a higher ranking in the container pool order is considered a more desirable candidate for removing containers.

[0171] The process removes idle containers from the container pools in a sorted order, without exceeding the maximum number of containers that can be removed from each container pool, until sufficient resources are available for adding containers (step 2006). In step 2006, the maximum number of containers that can be removed from each container pool, calculated in step 2002, is used. The process then terminates.

[0172] Transfer to Figure 21 This illustrates a flowchart of the process of performing job preemption according to an exemplary embodiment. Figure 21 The process in the middle can be replaced when using preemption. Figure 17 Example of an optional step to be performed in step 1712.

[0173] The process first identifies the lowest priority job in the container pool that is lower than the priority of the currently received job for processing (step 2100). It then determines whether there are lower priority jobs and whether the resources released by these lower priority jobs are sufficient to create a container for the current job (step 2102). If lower priority jobs exist and the resources released from them are sufficient, the process checks and preempts the lower priority jobs, placing them in a waiting queue (step 2104). In step 2104, the preempted jobs release resources for the current job.

[0174] The process uses the released resources to add a container with the job type of the current job (step 2106). Then, the process uses the added container to run the job (step 2108). The process then terminates.

[0175] Referring again to step 2102, if there are no lower priority jobs or insufficient resources to be released from lower priority jobs, the process then terminates.

[0176] The flowcharts and block diagrams in the different illustrated embodiments illustrate some possible architectures, functions, and operations of the apparatus and methods in the exemplary embodiments. In this regard, each block in a flowchart or block diagram may represent at least one of a module, segment, function, or part of an operation or step. For example, one or more blocks may be implemented as program instructions, hardware, or a combination of program instructions and hardware. For example, when implemented in hardware, the hardware may take the form of an integrated circuit manufactured or configured to perform one or more operations in the flowchart or block diagram. When implemented as a combination of program instructions and hardware, the implementation may take the form of firmware. Each block in a flowchart or block diagram may be implemented using a dedicated hardware system that performs different operations or combinations of dedicated hardware, as well as program instructions executed by the dedicated hardware.

[0177] In some alternative embodiments of the exemplary embodiments, one or more functions marked in the boxes may not appear in the order shown in the figures. For example, in some cases, two boxes displayed consecutively may execute substantially concurrently, or sometimes they may execute in reverse order, depending on the functions involved. Furthermore, in addition to the boxes shown in the flowchart or block diagram, other boxes may be added.

[0178] Now go to Figure 22 This diagram illustrates a data processing system according to an exemplary embodiment. The data processing system 2200 can be used to implement... Figure 1 Cloud computing node 110 Figure 1 Personal digital assistants (PDAs) or smartphones 120A, desktop computers 120B, laptops 120C, and / or automotive computer systems 120N. Data processing system 2200 can be used to implement... Figure 2 The computer in the hardware and software layer 202 and Figure 3 The data processing system 2200 includes server computer 304, server computer 306, and client device 310. It can also be used to implement... Figure 4 The computer system 414 is shown in this example. In this example, the data processing system 2200 includes a communication framework 2202 that provides communication between the processor unit 2204, memory 2206, persistent memory 2208, communication unit 2210, input / output (I / O) unit 2212, and display 2214. In this example, the communication framework 2202 takes the form of a bus system.

[0179] Processor unit 2204 is used to execute instructions for software that can be loaded into memory 2206. Processor unit 2204 includes one or more processors. For example, processor unit 2204 may be selected from at least one of a multi-core processor, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a network processor, or other suitable types of processors. Furthermore, processor unit 2204 may be implemented using one or more heterogeneous processor systems in which a main processor and auxiliary processors coexist on a single chip. As another example, processor unit 2204 may be a symmetric multiprocessor system containing multiple processors of the same type on a single chip.

[0180] Memory 2206 and persistent memory 2208 are examples of storage device 2216. A storage device is any hardware capable of storing information, such as, but not limited to, data, program instructions in functional form, or other suitable information, which can be temporary, permanent, or both. In these examples, storage device 2216 may also be referred to as a computer-readable storage device. In these examples, memory 2206 may be random access memory or any other suitable volatile or non-volatile storage device. Persistent memory 2208 can take many forms depending on the specific implementation.

[0181] For example, persistent storage 2208 may include one or more components or devices. For example, persistent storage 2208 may be a hard disk drive, a solid-state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination thereof. The media used in persistent storage 2208 may also be removable. For example, a removable hard disk drive may be used for persistent storage 2208.

[0182] In these examples, communication unit 2210 provides communication with other data processing systems or devices. In these examples, communication unit 2210 is a network interface card.

[0183] Input / output unit 2212 allows data input and output with other devices that can be connected to data processing system 2200. For example, input / output unit 2212 can provide connectivity for user input via at least one of a keyboard, mouse, or other suitable input device. Furthermore, input / output unit 2212 can send output to a printer. Display 2214 provides a mechanism for displaying information to the user.

[0184] Instructions for at least one of the operating system, application, or program may reside in a storage device 2216 that communicates with the processor unit 2204 via the communication framework 2202. The processor unit 2204 may execute processes of different embodiments using computer-implemented instructions that may reside in memory (e.g., memory 2206).

[0185] These instructions are referred to as program instructions that can be read and executed by the processor in processor unit 2204, computer-usable program instructions, or computer-readable program instructions. The program instructions in different embodiments may be embodied on different physical or computer-readable storage media, such as memory 2206 or persistent memory 2208.

[0186] Program instructions 2218 are functionally located on a selectively removable computer-readable medium 2220 and can be loaded or transferred to a data processing system 2200 for execution by a processor unit 2204. In these examples, program instructions 2218 and computer-readable medium 2220 constitute a computer program product 2222. In an exemplary example, computer-readable medium 2220 is a computer-readable storage medium 2224.

[0187] Computer-readable storage medium 2224 is a physical or tangible storage device for storing program instructions 2218, and not a medium for propagating or transmitting program instructions 2218. Computer-readable storage medium 2224 as used herein should not be construed as a transient signal, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0188] Alternatively, program instructions 2218 may be transmitted to data processing system 2200 using a computer-readable signal medium. The computer-readable signal medium is a signal, such as a propagated data signal containing program instructions 2218. For example, the computer-readable signal medium may be at least one of electromagnetic signals, optical signals, or any other suitable type of signal. These signals may be transmitted via connections such as wireless connections, fiber optic cables, coaxial cables, wires, or any other suitable type of connection.

[0189] Furthermore, as used herein, "computer-readable medium 2220" can be singular or plural. For example, program instructions 2218 may reside in a computer-readable medium 2220 in the form of a single storage device or system. In another example, program instructions 2218 may reside in computer-readable media 2220 distributed across multiple data processing systems. In other words, some instructions in program instructions 2218 may reside in one data processing system, while other instructions in program instructions 2218 may reside in another data processing system. For example, a portion of program instructions 2218 may reside in a computer-readable medium 2220 in a server computer, while another portion of program instructions 2218 may reside in a computer-readable medium 2220 located in a group of clients.

[0190] Therefore, exemplary embodiments provide a computer-implemented method, a computer system, and a computer program product for managing jobs using job processing pools. The computer system receives a job having a certain job type. The computer system identifies a job processing pool within the job processing pools for running jobs of that job type, wherein the job processing pool includes a job processor for running jobs of that job type, and wherein another job processing pool within the job processing pool contains other job processors for running jobs of different job types. The computer system uses the job processor of that job type in the job processing pool for that job type to run the job of that job type. According to other exemplary embodiments, a computer system and a computer program product for managing jobs are provided.

[0191] Therefore, in the illustration of the example, one or more technical solutions can provide technical effects that improve the performance of running jobs in a computer system, including at least one of the following: increasing the total throughput of job processing; reducing the waiting time for running jobs; or reducing the resources held by the job processor at any given time. In this example, the total throughput of running jobs can be increased over different time intervals. Furthermore, the waiting time between different types of jobs can be reduced. In other words, the job processor can be made available immediately or as quickly as possible for running jobs. Additionally, the container resources held by different containers at any given time can be reduced. Moreover, these efficiencies can be taken into account when jobs are running.

[0192] The descriptions of different exemplary embodiments are provided for illustrative and descriptive purposes and are not intended to be exhaustive or limited to the embodiments disclosed. The different examples describe components that perform actions or operations. In exemplary embodiments, components may be configured to perform said actions or operations. For example, a component may be configured or designed to provide the component with the ability to perform the actions or operations described in the examples as being performed by the component. Furthermore, for the purposes of the use of the terms “comprising,” “including,” “having,” “containing,” and variations thereof herein, such terms are intended to be inclusive, similar to the term “comprising” as an open transitional term, without excluding any additional or other elements.

[0193] Various embodiments of the invention have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Not all implementations will include all the features described in the illustrative examples. Furthermore, different illustrative embodiments may provide features that differ from other illustrative embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to the technology in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method for managing jobs using a job processing pool, the computer-implemented method comprising: The computer system receives a job with a specific job type. The computer system identifies job processing pools in the job processing pool for running jobs of that job type, wherein each job processing pool in the job processing pool processes jobs of the same job type, and two job processing pools in the job processing pool are duplicate job processing pools for running jobs of the same job type, wherein each job processing pool includes a job processor for running jobs of that job type, and wherein the other job processing pool in the job processing pool includes other job processors for running jobs of different job types; The computer system determines whether a job processor of that job type can be added to the job processing pool to run jobs of that job type; In response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type, and that the number of job processors of that job type added to the job processing pool does not exceed the resource limit, the computer system adds a job processor of that job type to the job processing pool; and The computer system uses the job processor of this job type to run the job.

2. The computer-implemented method according to claim 1, wherein, The computer system runs the job using the job processor of that job type in the job processing pool for that job type, including: The computer system responds to the fact that the job processor in the job processing pool for that job type is available to run the job, and the job is run by the job processor of that job type in the job processing pool for that job type.

3. The computer-implemented method according to claim 1, wherein, The addition of a job processor of a certain job type by a computer system in response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type includes: The computer system removes a group of idle job processors to free up resources for adding said job processors; The computer system adds a job processor for this job type to run jobs with this job type.

4. The computer-implemented method according to claim 1, wherein, The addition of a job processor of a certain job type by a computer system in response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type includes: The computer system sorts the job processing pool in order based on the total job priority of the job processing pool and the determination of the job processors in the job processing pool, according to the new resource requirements of the job processing pool that need to be increased in size based on time-based job prediction. The computer system removes idle job processors from the job processing pool with higher order ranking, so that sufficient resources are available to add the job processors. The computer system adds the job processor of this job type to the job processing pool.

5. The computer-implemented method according to claim 1, wherein, The addition of a job processor of a certain job type by a computer system in response to determining that a job processor of that job type can be added to the job processing pool to run jobs of that job type includes: The computer system identifies a group of candidate jobs with lower priority than the current job, wherein the resources freed up by transferring the group of candidate jobs are sufficient to add a job processor for the current job type; The computer system places this group of candidate jobs in a waiting queue; The computer system adds the job processor of this job type to the job processing pool.

6. The computer-implemented method according to claim 1, further comprising: The computer system removes idle job processors that have been idle for more than a threshold amount of time.

7. The computer-implemented method according to claim 1, further comprising: The computer system determines the time-based job forecast for the task; The size of the job processing pool is coordinated by a computer system using time-based job predictions.

8. The computer-implemented method according to claim 7, wherein, The computer system coordinates the size of the job processing pool using time-based job prediction. The computer system removes the current job processor from the job processing pool that needs to be reduced in size based on time-based job forecasts; The computer system adds new job processors to the job processing pool based on a set of time-based job predictions and resource constraints.

9. The computer-implemented method according to claim 8, wherein, Adding new job processors to the job processing pool by a computer system based on a set of time-based job forecasts and resource constraints includes: The computer system determines the additional resource requirements for the job processing pool, which needs to be increased in size based on time-based job forecasts. In response to determining that adding a new job processor to the job processing pool does not exceed the set of resource limits, the computer system adds a new job processor; In response to determining a set where adding new job processors to the job processing pool exceeds resource limits, the computer system sorts the job processing pool in order based on the total job priority of the job processing pool and the resource requirements of the job processors in the job processing pool according to time-based job predictions for the jobs. The computer system adds new job processors to the job processing pool in the order provided that the set of resource limits does not exceed a set of resource limits, wherein the set of resource limits is selected from one of global resource limits, consumer resource limits, or per-container-type limits.

10. The computer-implemented method according to claim 1, further comprising: In response to a situation where a current job in a job processor within a job processing pool has been running for a period exceeding a running time threshold, the computer system transfers the current job from the job processor within the job processing pool to a dedicated job processor outside the job processing pool.

11. The computer-implemented method according to claim 1, wherein, The job processor is selected from at least one of the following: container, pod, thread, process, application, operating system instance, virtual machine, host machine, or cluster.

12. A computer-implemented method for managing jobs using a job processing pool, the computer-implemented method comprising: In response to receiving a job of a certain job type, the computer system determines whether there is a container pool for that job type to run the job of that job type, wherein each container pool in the container pool runs the job of the selected job type, and there are two container pools in the container pool that are duplicate job processing pools for running jobs of the same job type. The computer system responds to the availability of the container by running the job using the container; In response to the unavailability of a container for that job type, the computer system determines whether a container for that job type can be added within a set not exceeding the resource limits. The computer system adds a container for that job type to run the job in response to determining that a set of resource limits can be added for that job type. The computer system places the job in a waiting queue in response to the determination that no container for that job type can be added if the set of resource limits is not exceeded.

13. A computer system, comprising: A plurality of processor units, wherein the plurality of processor units execute program instructions to perform the method according to any one of claims 1 to 12.

14. A computer program product for managing jobs using a job processing pool, the computer program product having program instructions embodied therein, the program product being executable by a computer system to cause the computer system to perform the method according to any one of claims 1 to 12.

Images