It operation management device and method

Abstract

To appropriately manage resources.SOLUTION: An environment construction device 20 selects a container to be preempted according to the characteristics of the workloads of a plurality of containers in an operation environment 30. The environment construction device 20 comprises a resource management unit 22, an operating information acquisition unit 23, and an environment information storage unit 24. The resource management unit manages the arrangement of containers in the operation environment as container arrangement information 242. The operating information acquisition unit acquires the operating situation of resources operating in the operation environment as container monitoring information 243. The environment information storage unit stores the container arrangement information and the container monitoring information. The resource management unit infers the workload characteristics of the containers under execution on the basis of the container arrangement information and the container monitoring information and selects a container to be preempted from the plurality of containers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an IT operations management apparatus and method. [Background technology]

[0002] In the field of artificial intelligence (AI), especially deep learning, high-performance graphics processing units (GPUs) are required for large-scale data processing and parallel computing. Because GPUs are expensive, efficient resource allocation and job scheduling are important in infrastructure shared by multiple development projects. On the other hand, central processing units (CPUs) are generally treated as more cost-effective resources, and their appropriate utilization is also important.

[0003] Although Kubernetes is used for resource scheduling for general web service workloads, it does not fully address the needs of AI development. To fill this gap, an open source software called Kubernetes was developed to address the specific scheduling needs of AI development (Non-Patent Document 1).

[0004] Furthermore, Patent Document 1 describes a method for determining whether to use a GPU based on the content of code, with the aim of efficiently using a CPU and a GPU. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Kueue, Internet<URL:https: / / kueue.sigs.k8s.io> [Patent Document 1] International Publication No. 2022 / 079748 Summary of the Invention [Problem to be solved by the invention]

[0006] Focusing on the problem of job scheduling in GPU infrastructure, the job scheduling system of Non-Patent Document 1 allocates resources to multiple container programs based on their priorities. During this process, a low-priority container program may be temporarily stopped to secure the resources required by a high-priority container program. This operation is called preemption. However, if a low-priority container program is stopped due to preemption, the throughput of that program will drop to zero.

[0007] Furthermore, in Patent Document 1, the content of the source code of the program to be deployed is analyzed to determine whether GPU usage is required. However, this determination method cannot identify the type of infrastructure to be used if the content of the program cannot be analyzed. This makes it difficult to dynamically and efficiently select and allocate resource types.

[0008] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technique for appropriately managing resources. [Means for solving the problem]

[0009] In order to achieve the above object, the present invention provides an IT operations management device that selects a container to be preempted according to the workload characteristics of each of a plurality of containers in an operational environment, and includes a resource management unit that manages the placement of the containers in the operational environment as container placement information, an operation information acquisition unit that acquires the operation status of resources running in the operational environment as container monitoring information, and an environment information storage unit that stores the container placement information and the container monitoring information, and the resource management unit infers the workload characteristics of running containers based on the container placement information and the container monitoring information, and selects a container to be preempted from the plurality of containers. [Effects of the Invention]

[0010] According to the present invention, resources can be managed appropriately. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 is a functional block diagram showing a configuration example of the environment construction device according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a preemption candidate table according to the first embodiment. [Figure 3] 3A to 3C are diagrams illustrating an example of container placement instruction information and container placement information according to the first embodiment. [Figure 4] 3A and 3B are diagrams illustrating an example of container monitoring information and node monitoring information according to the first embodiment. [Figure 5] FIG. 1 is a block diagram illustrating an example of an operating environment that does not use a hypervisor according to a first embodiment. [Figure 6] FIG. 1 is a block diagram illustrating an example of an operating environment using a hypervisor according to a first embodiment. [Figure 7] 10 is a flowchart showing an example of a container arrangement process according to the first embodiment. [Figure 8] 3A to 3C are diagrams illustrating an example of container placement instruction information and container placement information according to the first embodiment. [Figure 9] 10 is a flowchart illustrating an example of a preemption target selection process according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of container placement instruction information according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] This embodiment describes a method and system for optimizing the use of GPU (Graphical Processing Unit) resources and improving the operational management of IT systems. Specifically, the description focuses on the preemption decision process for GPU resources. The following description explains the system configuration and operating principles for efficiently utilizing infrastructure, as well as the program classification and scheduling method based on the system.

[0013] For example, we classify GPU-enabled programs into two categories: those that absolutely must use a GPU (hereafter referred to as "GPU-required"), and those that can improve performance by using a GPU but can still provide a usable response without using a GPU (hereafter referred to as "GPU-preferred").

[0014] In order to secure the GPU resources necessary for the placement of new container programs, when selecting existing container programs to preempt, GPU-preferred container programs are given priority.

[0015] Additionally, preempted GPU-preferred container programs are relocated without using the GPU, allowing these programs to continue running at a consistent level of performance even in situations where GPU resources are scarce.

[0016] The container program used here is designed to operate adaptively regardless of the presence or absence of a runtime environment. This program has the functionality to utilize the computing resources of a GPU if one is present, and to perform processing using an alternative computing means such as a CPU if a GPU is not present.

[0017] Hereinafter, examples will be described with reference to the drawings. Note that the following is merely an example of implementation, and the invention itself is not intended to be limited to the specific contents below.

[0018] Furthermore, the following description of the embodiments and the configurations and processes shown in the drawings are intended to provide an overview of the embodiments to the extent necessary for understanding and implementing the present invention, and are not intended to limit the embodiments of the present invention. Furthermore, the embodiments and their modifications can be combined in part or in whole within a range that is consistent with each other without departing from the spirit of the present invention. Example 1

[0019] In this embodiment, it is assumed that a request is made to place a new container program that uses a GPU when there are no free GPU resources. In this case, this embodiment shows a process of selecting a container program that does not cause any practical problems even if it does not use a GPU from among the currently running container programs as a preemption target, and then relocating the selected container program without using the GPU. <System Description>

[0020] FIG. 1 is a diagram illustrating an example of a functional configuration of an environment construction apparatus according to a first embodiment.

[0021] The IT operations management system 1 includes a user terminal 10, an environment construction device 20 as an example of an "IT operations management device", and an operational environment 30. The environment construction device 20 in Fig. 1 can deploy a container program in the operational environment 30 when container deployment instruction information is transmitted via the user terminal 10 by a user who wants to deploy a container program in operation.

[0022] The environment construction device 20 includes a placement information acquisition unit 21, a resource management unit 22, an operation information acquisition unit 23, and an environment information storage unit 24.

[0023] The placement information acquisition unit 21 can store container placement instruction information input from the user terminal 10 operated by the user in the environment information storage unit 24. The container placement instruction information will be described in detail later with reference to FIG. 3.

[0024] The resource management unit 22 includes a container placement function 221 , a preemption target selection function 222 , and a preemption target candidate table 223 .

[0025] The container placement function 221 can allocate necessary resources from the operational environment 30 to a container program based on the container placement instruction information 241 stored in the environment information storage unit 24, and execute the container program in the operational environment 30. The container placement function 221 refers to a known resource scheduling process.

[0026] The preemption target selection function 222 executes a distinctive preemption target selection process. In the preemption target selection process, the criteria for selecting a target to be preempted are not only the priority of the container, but also the container's characteristics, such as whether the container can return a usable response even when using a processor other than a GPU (an example of a "first processor") (GPU-preferred). The preemption target selection process will be explained in detail later using Figure 9.

[0027] The preemption target candidate table 223 is data that is temporarily used in the preemption target selection process.

[0028] FIG. 2 is a diagram illustrating an example of a preemption candidate table according to the first embodiment.

[0029] The preemption target candidate table 223 includes a priority level, a node ID, a container ID, node resource information, and a relocation target flag.

[0030] The priority level is a value related to the execution priority. The node ID is an identifier for uniquely identifying the node on which the container that is a candidate for preemption is located. The container ID is an identifier for uniquely identifying the container. The node resource information indicates the resources on the node that each container is using. The relocation target flag indicates that the container will be relocated without using the GPU after preemption.

[0031] Returning to Fig. 1, the operation information acquisition unit 23 acquires the operation status of resources operating in the operation environment 30 as container monitoring information 243. Specifically, the operation information acquisition unit 23 periodically acquires the container monitoring information 243 of the operation environment 30 and stores information on the computing nodes running the containers and the resources used by the containers in the operation information storage unit 24. The operation information acquisition unit 23 pays particular attention to the resource information used by the containers and stores it as the container monitoring information 243.

[0032] FIG. 3 is a diagram illustrating an example of container placement instruction information and container placement information, and FIG. 4 is a diagram illustrating an example of container monitoring information and node monitoring information according to the first embodiment.

[0033] As shown in FIGS. 3 and 4, the environment information storage unit 24 includes container placement instruction information 241, container placement information 242, container monitoring information 243, and a node monitoring table 244.

[0034] The container placement instruction information 241 is data that contains conditions such as the number of containers to be executed and required resources desired by the user, and is transmitted to the placement information acquisition unit 21 via the user terminal 100. The container placement instruction information 241 includes an ID, a service name, required resources, a container image, a priority level, a placement option, and a post-relocation instruction information ID.

[0035] The id is an identifier for uniquely identifying the container placement instruction information 241. The service name is an identifier that allows the user to uniquely identify the processing content they want to execute in the container. The required resources represent the amount of GPU, CPU (Central Processing Unit) as an example of a "second processor," memory, etc. required to execute the container. The container image is an identifier for uniquely identifying the container to be executed. The priority level is a value related to the execution priority. The placement option represents the number of executions and whether or not a restart is required in the event of an error. The post-relocation instruction information id represents history information in which the placement instruction has been changed by preemption accompanied by relocation. The priority level can be smaller the lower the priority, and larger the higher the priority.

[0036] The container placement information 242 is data related to containers placed in the operational environment 30. The container placement information 242 includes a container ID, a container name, placement destination information, a placement instruction information ID, and a priority level.

[0037] The container ID is an identifier that uniquely identifies a container instance. The container name is an identifier that allows the user to uniquely identify the processing content of the container. The placement destination information indicates the node where the container instance is placed and the identifier of the resources used on that node. The placement instruction information ID indicates the basis for placement. The priority level is a value related to the execution priority.

[0038] The container monitoring information 243 is data obtained by the operation information acquisition unit 203 monitoring the container programs deployed and executed in the operational environment 30. The container monitoring information 2043 includes a container ID and monitoring information.

[0039] The container ID is an identifier for uniquely identifying the container to be monitored. The monitoring information is time-series data of the results of monitoring the container to be monitored. The monitoring information may include the node where the container is located, the amount of resources used by the container, the average response time of requests being processed by the container, and the timestamp information of the time when the monitoring information was obtained.

[0040] The node monitoring information 244 is data obtained by monitoring the compute nodes that make up the operational environment 30 by the performance information acquisition unit 23. The node monitoring information 2044 includes a node ID and free resources.

[0041] The node ID is an identifier that uniquely identifies a compute node. The node spec indicates the amount of resources that the node possesses. The free resources indicate unused resources that have not yet been reserved for placing containers on the compute node.

[0042] FIG. 5 is a block diagram illustrating an example of an operational environment that does not use a hypervisor according to the first embodiment, and FIG. 6 is a block diagram illustrating an example of an operational environment that uses a hypervisor according to the first embodiment.

[0043] 5 and 6, the operating environment 30 includes computing nodes 50 and 60 that use virtualization technology. The operating environment 30 uses either of these types of compute nodes 50 and 60. Furthermore, the operating environment 30 may be configured by combining one or more compute nodes to deploy and run containers.

[0044] As shown in FIG. 5, each compute node 50 includes a number of pieces of hardware. This hardware may include one or more CPUs, GPUs, memory, network interface cards (NICs), and storage disks (disk drives). The disk drives may include solid-state drives, hard disk drives, or some combination of the two. The compute nodes 50 run a host operating system on the hardware. One or more container programs run on the host operating system.

[0045] Similarly, as shown in FIG. 6 , each compute node 60 includes a number of pieces of hardware. This hardware may include one or more GPUs, CPUs, memory, network interface cards (NICs), and storage disks (disk drives). The disk drives may include solid-state drives, hard disk drives, or some combination of the two. The compute nodes 60 run a host operating system on the hardware. The compute nodes 60 also include a hypervisor for sharing and managing the hardware, allowing multiple different virtual machines that are isolated from each other to run on the same compute node (physical machine) 60. Each compute node 60 includes one or more virtual machines, each of which may include a guest operating system and one or more container programs that run on the guest operating system.

[0046] Next, a flow of container placement in the operational environment 30 in the resource management unit 22 in this embodiment will be described with reference to FIG.

[0047] FIG. 7 is a flowchart illustrating an example of a container placement process according to the first embodiment, and FIG. 8 is a diagram illustrating an example of container placement instruction information and container placement information according to the first embodiment.

[0048] The resource management unit 22 periodically starts the processing shown in Fig. 7. First, the resource management unit 22 references the container placement instruction information 241 and the container placement information 242 stored in the operation information storage unit 24, and determines whether there is any container program for which placement has not been completed (S701). If the determination result in S701 is false (S701: NO), that is, if all container placement has been completed, the resource management unit 22 ends the processing as is and waits for the timing of the next periodic execution. If the determination result in S701 is true (S701: YES), that is, if there is one or more pieces of placement instruction information that have not been executed, the resource management unit 22 transitions to S702.

[0049] Next, in S702, the resource management unit 22 selects one of the unexecuted placement instruction information, and the process proceeds to S703. In the example of Fig. 3, the resource management unit 22 selects the container placement instruction dp-010, whose placement is incomplete, in the row 2411 of the container placement instruction information 241, and the process proceeds to S703.

[0050] Next, in S703, the resource management unit 22 refers to the container placement information 242 and the node monitoring information 244 and determines whether there are free resources necessary to start a new container, i.e., whether a node exists that can satisfy the requested resources. If the determination result in S703 is true (S703: YES), i.e., if free resources that satisfy the request exist, the resource management unit 22 transitions to S708. If the determination result in S703 is false (S703: NO), i.e., if free resources that satisfy the request do not exist, the resource management unit 22 transitions to S704. In the example of FIG. 3, the placement instruction information dp-010 requests resources for two containers, including one GPU, two CPUs, and 8 GB of memory per container. However, there are no free resources that satisfy the request (node ​​monitoring information 244 in FIG. 4). In this case, it is necessary to determine whether to perform preemption, so the resource management unit 22 transitions to S704.

[0051] Next, in S704, the resource management unit 22 executes the preemption target selection function 222 (preemption target selection process). In the example of FIG. 3, the resource management unit 22 receives requested resource information of placement instruction information dp-010, such as "resources of two GPUs, two CPUs, and 16 GB of memory per container," and priority information of "priority 100." As a result, the resource management unit 22 outputs containers Cnt-003 and Cnt-004 as preemption targets. A detailed description of this preemption target selection process will be given with reference to FIG. 9.

[0052] Next, in S705, the resource management unit 22 refers to the execution result of the preemption target selection function 222 and determines whether there is a preemption target. If the determination result in S705 is false (S705: NO), it means that there are no resources to create a new container and it is not possible to create free resources. In this case, the resource management unit 22 ends this process shown in FIG. 7 and waits for the timing of the next periodic execution. If the determination result in S705 is true (S705: YES), the resource management unit 22 transitions to S706. In the example of FIG. 3, the resource management unit 22 transitions to S706 because containers Cnt-003 and Cnt-004 exist as preemption targets.

[0053] Next, in S706, the resource management unit 22 changes the placement instruction information of the container that is to be rearranged among one or more preemption target containers. In Fig. 3, the preemption target containers are two containers Cnt-003 and Cnt-004. By referring to row 2421 of the container placement information 242 in Fig. 3, it can be seen that the original placement instruction information of this container is placement instruction information dp-002.

[0054] New container placement instruction information dp-011 is created by removing the GPU-related resources from the required resources in this placement instruction information dp-002 (row 2412 of the container placement instruction information 241 in FIG. 8). Furthermore, the post-relocation instruction information id of the original placement instruction information dp-002 is updated to dp-011. As a result, the placement instruction information dp-002 is excluded from the search targets for incompletely placed containers in S701.

[0055] Next, in S707, the resource management unit 22 preempts the container that is the target of preemption. At this point, container Cn-003 is stopped and deleted. At this time, the operation information acquisition unit 23, which monitors the operational environment 300 independently of the resource management unit 202, detects that containers Cnt-003 and Cnt-004 have been deleted, and deletes row 2422 of the container placement information 242 (FIG. 8).

[0056] Next, in S708, the container placement function 221 of the resource management unit 22 executes container placement processing based on the container placement instruction information selected in S702. In the example of Fig. 3, the resource management unit 22 places containers Cnt-023 and Cnt-024 in the operational environment 30 based on the placement instruction information dp-010. At this time, the operation information acquisition unit 23, which monitors the operational environment 30 independently of the resource management unit 202, detects that two new containers have been created, and adds lines 2423 and 2424 to the container placement information 242 (Fig. 8).

[0057] Next, the flow of the preemption target selection process by the preemption target selection function 222 of the resource management unit 22 will be described with reference to FIG.

[0058] FIG. 9 is a flowchart illustrating an example of the preemption target selection process according to the first embodiment.

[0059] The preemption target selection process is executed when new container placement instruction information is found to search for a container to be stopped or deleted from among the currently running containers in order to secure resources. In S704 of FIG. 7, the preemption target selection function 222 receives, as input, required resource information and priority level information of the container program to be newly placed. Then, in S901, the preemption target selection function 222 creates a preemption target candidate table 223 while referencing the container placement information 242, container monitoring information 243, and node monitoring information 244. In the example of FIG. 3, the required resource information of placement instruction information dp-010, "resources of two GPUs, two CPU cores, and 16 GB of memory per container," and priority information of "priority 100" are input.

[0060] The preemption target selection function 222 extracts the container IDs and placement destination information of containers that use "GPU, CPU, memory" and have a low priority level lower than "10" from the container information described in the container placement information 242. As a result, a table including containers Cnt-001 to Cnt-005 is created. A column for the relocation target flag is added to this table, and the table is sorted in order of priority and placement destination node to create the preemption candidate table 2023.

[0061] Next, in S902, the preemption target selection function 222 identifies the amount of resources required per container from the requested resources, and starts a loop process that repeats S904 and subsequent steps for the required number of containers. In this embodiment, it is sufficient to reserve resources once to place a container with "two GPUs, two CPU cores, and 16 GB of memory."

[0062] Next, in S903, the preemption target selection function 222 determines whether the contents of the preemption target candidate table 223 are empty. If the determination result in S903 is true (S903: YES), the preemption target selection function 222 transitions to S912 because resources cannot be secured by preemption. If the determination result in S903 is false (S903: NO), the preemption target selection function 222 transitions to S904. In the example of Fig. 3, the preemption target candidate table 223 is not empty, so the function transitions to S904.

[0063] Next, in S904, the preemption target selection function 222 selects one target from the preemption target candidate table 223 and starts processing from S905 onwards. The selection criterion at this time is to basically select one row at a time from the top of the table that does not have a reallocation target flag filled in. In the example of Figure 3, this is executed in the order of Cnt-001, Cnt-003, Cnt-002, ...

[0064] Next, in S905, the preemption target selection function 222 refers to the container monitoring information 243 and determines whether the container selected in S904 uses a GPU but can return a practical response without using a GPU (GPU-preferred). If the determination result in S905 is true (S905: Yes), the preemption target selection function 222 enables the relocation target flag and transitions to S906. If the determination result in S905 is true (S905: No), the preemption target selection function 222 deletes the selected row from the preemption candidate table 223 and transitions to S904. In the example of FIG. 3, the container that is the target for first executing the processing of S905 is container Cnt-001. Referring to the container monitoring information 243, the time series data from monitoring container Cnt-001 shows that the AverageResponseTime, which indicates the average request processing time, is greater than 30 seconds, and that the GPU utilization rate is greater than 75%.

[0065] Generally, the difference in processing performance between a GPU and a CPU results in a difference in response performance of several to several thousand times. Therefore, if a process that has a high GPU utilization rate but a response time of tens of seconds is processed only by the CPU without using the GPU, practical response performance cannot be provided to users who send requests to that container. Therefore, it can be determined that the Cnt-001 container is not a GPU-preferred container. Therefore, the preemption target selection function 222 deletes row 2251, transitions to S904, and performs a similar check on the next preemption candidate container. On the other hand, the container Cnt-003, which is confirmed in S905 after the Cnt-001 container, has a GPU utilization rate of no more than 65% and an average response time of approximately 3 ms. Since this process requires an extremely short GPU processing time, it is highly likely that practical response times can be achieved even if it is processed only by the CPU without using the GPU. Therefore, it can be determined that the container is GPU-preferred, and the relocation target flag is set to Yes, and the process transitions to S906. The state of the preemption candidate table 223 at this point is shown in the preemption candidate table 224 in FIG.

[0066] Next, in S906, the preemption target selection function 222 determines whether the required number of free GPUs can be secured if all valid containers are deleted. If the determination result in S906 is true (S906: Yes), the preemption target selection function 222 transitions to S909. If the determination result in S906 is false (S906: No), the preemption target selection function 222 transitions to S907. In the example of FIG. 3, at this point in time, only Cnt-003 has an enabled relocation target flag in the preemption candidate table 242. Checking the node resource information for Cnt-003 reveals that if Cnt-003 is deleted, at least "one GPU and one CPU" will be released. Furthermore, when checking the free resource information of node-002 that can be read from the container monitoring information 243 and the node monitoring information 244, it is found that even if container Cnt-003 is deleted, it is not possible to secure resources in node-002 that satisfy the request of "two GPUs, two CPU cores, and 16 GB of memory per container." Therefore, the preemption target selection function 222 transitions to S907.

[0067] Next, in S907, the preemption target selection function 222 determines whether there is a container in the preemption target candidate table 223 that is running on the same compute node as the container selected in S904. If the determination result in S907 is true (S907: Yes), the preemption target selection function 222 transitions to S908. If the determination result in S907 is false (S907: No), the preemption target selection function 222 transitions to S911. In the example of FIG. 3, at this point, Cnt-004 is placed on the same node (node-001) as container Cnt-003, and is listed in the preemption candidate table 2024 as a container whose relocation target flag is not yet entered. Therefore, the preemption target selection function 222 transitions to S908.

[0068] In S908, the preemption target selection function 222 transitions to S904 with the container discovered in S907 as the next container, and repeats the processing from S905 onwards. In the example of FIG. 3, container Cnt-004 was discovered in S907, so the preemption target selection function 222 repeats the processing from S905 onwards for Cnt-004. Since Cnt-004 is a container instance based on the same container placement instruction information 241 as Cnt-003, in the example of FIG. 3, it is assumed that the same processing characteristics as Cnt-003 were read from the monitoring information 243. In this case, the result of S905 is "Yes," and the result of S906 is "Can be secured." The state of the preemption target candidate table 223 at the completion of S906 is shown in the preemption candidate table 2025 of FIG. 2.

[0069] Next, in S909, the preemption target selection function 222 determines whether the loop processing started in S902 has ended. That is, the preemption target selection function 222 determines whether the resources required to place the required number of containers have been secured. If the determination result in S909 is true (S909: Yes), the preemption target selection function 222 transitions to S910. If the determination result in S909 is false (S909: No), the preemption target selection function 222 transitions to S902 and searches for resources for the next container. In the example of FIG. 3, the requested resources are "two GPUs per container, two CPU cores, and 16 GB of memory per container." Therefore, the request can be met by preempting containers Cnt-003 and Cnt-004. Therefore, the preemption target selection function 222 transitions to S910.

[0070] Next, in S910, the preemption target selection function 222 outputs rows for which the rearrangement target flag is enabled as a preemption target container list, which is the execution result of the preemption target selection function 222. In the example of Fig. 3, rows 2252 and 2253 relating to containers Cnt-003 and Cnt-004 in the preemption target candidate table 223 are output as the preemption target container list.

[0071] As described above, according to this embodiment, the environment construction device 20 selects a container to be preempted in accordance with the workload characteristics of each of a plurality of containers in the operational environment 30. The environment construction device 20 includes a resource management unit 22, an operation information acquisition unit 23, and an environment information storage unit 24. The resource management unit 22 manages the placement of containers in the operational environment 30 as container placement information 242. The operation information acquisition unit 23 acquires the operation status of resources running in the operational environment 30 as container monitoring information 243. The environment information storage unit 24 stores the container placement information 242 and the container monitoring information 243. The resource management unit 22 estimates the workload characteristics of the running containers based on the container placement information 242 and the container monitoring information 243, and selects a container to be preempted from a plurality of containers.

[0072] This prevents the throughput of the preempted program from dropping to zero, allowing for appropriate resource management. Furthermore, because preempted GPU-preferred programs are relocated without using the GPU, these programs continue to run with consistent performance. This minimizes the degradation of overall system throughput.

[0073] Furthermore, there is no need to analyze the contents of the source code: the identification of whether a program is GPU-required or GPU-preferred is based on monitoring information of the environment in which the program is running, so it is possible to determine whether to use a GPU even if the contents of the program's source code are unknown.

[0074] Furthermore, it can effectively utilize surplus CPU resources. By using CPU resources to reallocate preempted GPU-preferred programs when GPU resources are insufficient, it can efficiently utilize unused CPU resources and improve the resource efficiency of the entire system.

[0075] Furthermore, the container monitoring information 243 includes the request processing time and GPU utilization rate for each of the multiple containers, and the resource management unit 22 selects a container to be preempted based on the request processing time and GPU utilization rate. This allows a container to maintain practical response performance even if it does not use a GPU. Example 2

[0076] In the first embodiment, we have explained the flow of how resources are allocated to containers with higher priority by preempting the running containers Cnt-003 and Cnt-004. Hereinafter, we will explain the flow of how resources cannot be secured by preemption alone because the requested resource information input to the preemption target selection process is too large.

[0077] Consider the case where dp-010 included in the container placement instruction information 241 requests "resources of eight GPUs, eight CPU cores, and 256 GB of memory per container, for two containers" (row 2412 in FIG. 10). The overall flow is the same as in the first embodiment. However, the difference is the point in time when S906 is processed after the relocation target flags for Cnt-003 and Cnt-004 become valid (preemption candidate table 225 in FIG. 2) in the preemption target selection process in FIG. 9. In this embodiment, the preemption target selection function 222 transitions to S907 because the result of S906 is "unable to secure."

[0078] In S907, the container that is placed on node node-001 and for which the relocation target flag is not input does not exist in the preemption target candidate table 223. Therefore, the preemption target selection function 222 transitions to S911.

[0079] Next, in S911, the preemption target selection function 222 deletes all container information that is placed on the same compute node as Cnt-004 from the preemption target candidate table 223. In the example of Fig. 2, rows 2251, 2252, and 2253 of the preemption candidate table 225 are deleted.

[0080] Next, in S903, the preemption target selection function 222 determines whether the preemption target candidate table 223 is empty. If the determination result of S903 is false (S903: NO), the preemption target selection function 222 transitions to S904. If the determination result of S903 is true (S903: Yes), the preemption target selection function 222 transitions to S912. In the example of FIG. 2, as a result of executing S911, rows 2254 and 2255 remain in the preemption candidate table 2025, so the preemption target selection function 222 transitions to S904. Thereafter, processing from S905 onwards is performed on Cnt-002 and Cnt-005. However, regardless of the state of these two containers, S906 never becomes "Yes," so S911 is executed in each process, and finally, a determination that the container is "empty" is made in S903.

[0081] Next, in S912, the preemption target selection function 222 outputs the empty preemption target container list as the execution result, that is, the preemption target container list.

[0082] The present invention is not limited to the above-described embodiments as they are, and in the implementation stage, the components can be modified and embodied within the scope of the gist of the present invention, or multiple components disclosed in the above-described embodiments can be appropriately combined.

[0083] For example, the container monitoring information 243 may include the VRAM usage amount for each of multiple containers, and the resource management unit 22 may select a container to be preempted based on the VRAM usage amount. This allows the container to maintain practical response performance even if it does not use a GPU. [Explanation of symbols]

[0084] 20... environment construction device, 21... placement instruction information acquisition unit, 22... resource management unit, 23... operation information acquisition unit, 24... environment information storage unit, 241... container placement instruction information, 242... container placement information, 243... container monitoring information, 30... operation environment

Claims

1. An IT operations management device that selects a container to be preempted according to workload characteristics of each of a plurality of containers in an operation environment, a resource management unit that manages the placement of the container in the operational environment as container placement information; an operation information acquisition unit that acquires operation statuses of resources operating in the operation environment as container monitoring information; an environmental information storage unit that stores the container placement information and the container monitoring information, The resource management unit is an IT operations management device that estimates the workload characteristics of running containers based on the container placement information and the container monitoring information, and selects a container to be preempted from the multiple containers.

2. a placement instruction information acquisition unit that acquires a placement instruction for the container with respect to the operating environment as container placement instruction information, the resource management unit selects the container to be preempted based on the container placement instruction information, the container placement information, and the container monitoring information; The IT operations management device according to claim 1 .

3. the resource includes a first processor and a second processor having a processing performance lower than that of the first processor; the resource management unit selects a container to be relocated from the first processor to the second processor as the preemption target; The IT operations management device according to claim 1 .

4. the first processor is a GPU; the second processor is a CPU; The IT operations management device according to claim 3 .

5. the container monitoring information includes a request processing time and a GPU utilization rate for each of the plurality of containers; the resource management unit selects the container to be preempted based on the request processing time and the GPU utilization rate; The IT operations management device according to claim 4.

6. The container monitoring information includes a VRAM usage amount for each of the plurality of containers, The resource management unit selects the container to be preempted based on the VRAM usage amount. The IT operations management device according to claim 1 .

7. An IT operations management method using an IT operations management device that selects a container to be preempted according to workload characteristics of each of multiple containers in an operating environment, comprising: acquiring a container placement instruction for the operational environment as container placement instruction information; acquiring a location of the container relative to the operational environment as container location information; acquiring the operating status of resources operating in the operating environment as container monitoring information; storing the container placement instruction information, the container placement information, and the container monitoring information; and a step of estimating workload characteristics of running containers based on the container placement information and the container monitoring information, and selecting a container to be preempted from the plurality of containers.

8. The step of selecting the container to be preempted includes placing the container to be preempted in the operating environment so that the container to be preempted is processed by a CPU among the resources without using a GPU.

8. The IT operations management method according to claim 7.

9. the container monitoring information includes a request processing time and a GPU utilization rate for each of the plurality of containers; the step of selecting a container to be preempted includes selecting the container to be preempted based on the request processing time and a GPU utilization rate; 9. The IT operations management method according to claim 8.

10. The container monitoring information includes a VRAM usage amount for each of the plurality of containers, The step of selecting a container to be preempted includes selecting the container to be preempted based on the VRAM usage amount.

8. The IT operations management method according to claim 7.

Images