Container resource scheduling method, device and computer program product

Abstract

The embodiment of the invention relates to a container resource scheduling method and device and a computer program product. The method includes detecting whether there is an idle set of virtual devices that are not allocated to one or more container groups in response to the absence of available virtual devices among the virtual devices allocated to the one or more container groups on the host. The method also includes, in response to the absence of the set of virtual devices, processing, using a CPU of the host, a workload of the one or more container groups. According to the scheme, the utilization of idle acceleration resources can be promoted during the fault period of the container group, and unnecessary CPU occupation can be avoided as far as possible, so that system resources can be utilized more reasonably and efficiently.

Description

Technical Field

[0001] Embodiments of this disclosure relate to virtualization technology, and more specifically, to methods, apparatus, and computer program products for scheduling container resources. Background Technology

[0002] The host machine can use the computing resources of its physical acceleration devices to handle container workloads, allowing the host's Central Processing Unit (CPU) to focus on global control, scheduling, and other tasks. Physical acceleration devices provide corresponding Physical Features (PFs). These physical acceleration devices are also simply referred to as PF devices. For illustrative purposes, the terms "physical acceleration device" and "PF device" or similar terms are used interchangeably in this document to refer to hardware devices that provide computing resources outside of the CPU (e.g., hardware accelerator cards in the host machine).

[0003] The host can utilize a device plug-in mechanism to manage and allocate physical acceleration device resources. Such plug-ins can map physical acceleration device resources to multiple Virtual Function I / O (VF I / O) devices. VF I / O devices are also simply referred to as Virtual Function (VF) devices. For clarity, the terms "virtual device" and "VF device" or similar terms are used interchangeably in this document. Summary of the Invention

[0004] The embodiments of this disclosure provide a scheme for scheduling container resources.

[0005] In a first aspect of this disclosure, a method for scheduling container resources is provided, comprising: in response to the absence of available virtual devices among the virtual devices allocated to one or more container groups on a host, detecting the existence of a set of idle virtual devices not allocated to the one or more container groups, wherein each virtual device is run by a physical acceleration device among a plurality of physical acceleration devices of the host; and in response to the absence of the set of virtual devices, using the host's central processing unit to process the workload of the one or more container groups.

[0006] In a second aspect of this disclosure, an electronic device is provided, including a processor and a memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions including: detecting the existence of a set of idle virtual devices not assigned to the one or more container groups in response to the absence of available virtual devices among the virtual devices allocated to one or more container groups on a host, wherein each virtual device is operated by a physical acceleration device among a plurality of physical acceleration devices on the host; and using the host's central processing unit to process the workload of the one or more container groups in response to the absence of the set of virtual devices.

[0007] In a third aspect of this disclosure, a computer program product is provided, which is tangibly stored on a computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to perform the method described in accordance with a first aspect of this disclosure.

[0008] Please note that this summary is provided to present a simplified version of the chosen concepts, which will be further described in the detailed embodiments below. This summary is not intended to identify key or principal features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0009] The above and other objects, features, and advantages of this disclosure will become more apparent from a more detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating an example scheduling method for container resources according to some embodiments of this disclosure is shown;

[0012] Figure 3 An example scenario of a physics acceleration device malfunction is shown;

[0013] Figure 4 It shows the Figure 3 Example scenario of container group after fault handling in example scenario;

[0014] Figure 5 This diagram illustrates the resource usage of a physics acceleration device on an example system.

[0015] Figure 6 It shows Figure 5 This is a diagram illustrating the resource usage of a physical acceleration device in an example system after a failure.

[0016] Figure 7 It shows in Figure 6 A diagram illustrating resource usage after a faulty physics acceleration device in an example system is restored; and

[0017] Figure 8 A schematic block diagram of a device that can be used to implement embodiments of the present disclosure is shown.

[0018] In all the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment". Definitions of other terms will be given in the description below.

[0021] As used in this article, depending on the context, "meeting the threshold" can mean that the value is greater than the threshold, greater than or equal to the threshold, equal to the threshold, less than or equal to the threshold, less than the threshold, etc.

[0022] The host machine can use the computing resources of its physical acceleration devices to handle container workloads, allowing the host's CPU to focus on global control, scheduling, and other tasks. An example of a physical acceleration device is a Quick Assist Technology (QAT) accelerator. Multiple virtual devices, or VF devices, can run on a single physical acceleration device. A VF device is a virtual instance created on a physical acceleration device using virtualization technology; the resources required for a VF device to run are actually provided by the associated physical acceleration device.

[0023] Using container technology, multiple container groups (PODs) can be created on a host machine. Each container group consists of one or more containers and runs as the smallest deployable and manageable computing unit. Containers within a container group work together and share various resources such as storage and networking. For example, containers providing specific services or applications can be organized into container groups. Each container group can be assigned a set of VF devices to execute the workload of that container group.

[0024] When a physical acceleration device fails or becomes unavailable / offline for other reasons, the VF devices on it also become unavailable. In traditional fault handling, a CPU fallback mechanism can be applied when a VF device becomes unavailable to temporarily offload the workload handled by that VF device to the host CPU. Because containerized applications are unaware of other VF devices on the host that are not assigned to them, the workload of a container group will be directly transferred to the CPU while all VF devices assigned to the container group become unavailable.

[0025] However, a host can include multiple physical acceleration devices (e.g., in the form of hardware accelerator cards). While a VF device assigned to a container group becomes unavailable due to the unavailability of its corresponding physical acceleration device, other physical acceleration devices on the host may still have available idle VF devices. However, the application within the container group itself is unaware of the existence of idle VF devices not assigned to it, and its workload is still allocated to the host's CPU through the CPU bounce mechanism, resulting in unnecessary CPU usage.

[0026] To at least partially address the aforementioned problems and other potential issues, embodiments of this disclosure propose a container resource scheduling scheme. When no available virtual devices exist in the virtual devices allocated to one or more container groups on a host, the scheme first detects whether there are any idle virtual devices not allocated to those container groups. Only when no such idle virtual devices are detected will the scheme utilize the host's CPU to handle the workload of those one or more container groups.

[0027] In this way, before the fault handling mechanism for the container group unavoidably occupies CPU resources, it is possible to find as many idle acceleration resources as possible globally to handle the workload of the failed container group. This promotes the full utilization of acceleration resources and avoids unnecessary CPU occupation, thereby making more rational and efficient use of system resources and reducing the impact of device failure on system performance. Some embodiments of this disclosure are well adapted to cluster environments for containerized applications.

[0028] Figure 1A schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, environment 100 includes host 110. As a non-limiting example, host 110 may be a node server within a platform of a service provider offering services such as data storage and / or data protection. Although shown as a single entity, host 110 may also exist in other forms, such as distributed systems, as long as it includes the components required for embodiments of this disclosure.

[0029] Host 110 includes a CPU (not shown) and a set of physical acceleration devices 120-1, 120-2, ..., and 120-M (individually or collectively referred to as physical acceleration device 120). Each physical acceleration device 120 can instantiate multiple virtual devices to provide computing resources for various workloads on host 110, such as data compression and decompression, encryption and decryption, etc. For example, physical acceleration device 120-1 has associated virtual devices 140-1 to 140-4, etc., and physical acceleration device 120-M has associated virtual devices 140-5 to 140-8.

[0030] Host 110 also runs multiple container groups 130-1, 130-2, ..., and 130-N (individually or collectively referred to as container group 130). Virtual devices 140 can be allocated as resources to a container group 130 to handle workloads for that container group 130. For example, container group 130-1 is allocated virtual devices 140-1 and 140-2, container group 130-2 is allocated virtual devices 140-3 and 140-4, and container group 130-N is allocated virtual devices 140-7 and 140-8. It should be understood that... Figure 1 The number of components shown is for illustrative purposes only; host 110 may include more or fewer such components. Furthermore, the number of other types of components associated with a component is also for illustrative purposes only. For example, container group 130 may be allocated more or fewer virtual devices 140.

[0031] The architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Other devices, systems, or components, not shown, may also be present in example environment 200. For example, host 110 may also include a control plugin for globally managing and scheduling acceleration device resources. This control plugin can map the resources of physical acceleration devices into multiple virtual devices and integrate them into a container orchestration platform for subsequent management and scheduling. For example, environment 110 may also include various terminal devices requesting various services from host 110. Furthermore, embodiments of this disclosure can be applied to other environments with different structures and / or functionalities.

[0032] Figure 2A flowchart of an example scheduling method 200 for container resources according to some embodiments of the present disclosure is shown. Example method 200 can be, for example, by... Figure 1 The host 110 shown is used to execute this. It should be understood that method 100 may also include additional actions not shown, and the scope of this disclosure is not limited in this respect. The following is in conjunction with... Figure 1 Example environment 100 is used to describe method 200 in detail.

[0033] In 210, in response to the fact that there are no available virtual devices among the virtual devices assigned to one or more container groups on the host, it is detected whether there is a set of idle virtual devices not assigned to the one or more container groups, wherein each virtual device is run by a physical acceleration device among a plurality of physical acceleration devices of the host.

[0034] For example, when there are no available virtual devices in the virtual devices 140 of one or more container groups 130 assigned to host 110 (e.g., due to failure or otherwise offline), host 110 may (e.g., using a control plugin for acceleration devices) detect the existence of a set of idle virtual devices not assigned to the one or more container groups, where each virtual device is run by a physical acceleration device in one of the host's multiple physical acceleration devices 120.

[0035] During the operation of host 110, physical acceleration device 120 may become unavailable due to malfunctions or other reasons, thereby making its associated virtual devices 140 unavailable. For example, when physical acceleration device 120-1 becomes unavailable, virtual devices 140-1 to 140-4 allocated to container groups 130-1 and 130-2 become unavailable. In this case, no virtual devices will be available to handle the workloads of container groups 130-1 and 130-2. To address this, host 110 can detect whether there are any virtual devices on its other physical acceleration devices 140 that have not been allocated to container groups.

[0036] At 220, in response to the absence of that set of virtual devices, the host's CPU is used to handle the workload of that one or more container groups. For example, if no such set of idle virtual devices is detected at 210, host 110 will use its CPU to handle the workloads of container groups 130-1 and 130-2 associated with the unavailable virtual devices. In other words, host 110 can apply a CPU bounce mechanism to redirect the workloads of container groups 130-1 and 130-2 to its CPU for execution. This allows the services of the container groups originally associated with a physical acceleration device to continue operating normally in the event of a partial physical acceleration device failure.

[0037] Method 200 optimizes the fault handling mechanism for container groups. By performing a global check before containerized applications unnecessarily consume host CPU resources, it is possible to identify idle acceleration resources available for the workload of handling faulty container groups, thereby avoiding unnecessary CPU consumption and enabling more efficient use of host resources.

[0038] In some embodiments, host 110 may detect the presence of a set of idle virtual devices. For example, in Figure 1 In the example, host 110 can detect idle virtual devices 140-7 and 140-8 on physical acceleration device 120-M. Then, host 110 can assign the detected set of virtual devices to at least one of the one or more container groups to handle the workload of that at least one container group.

[0039] In some cases, depending on the workload required to run the one or more container groups, there may not be enough idle virtual devices to allocate to all container groups within those one or more container groups. In some such embodiments, after all idle virtual devices have been allocated, host 110 can use its CPU to handle the workload of other container groups within the one or more container groups besides the at least one container group; that is, a CPU rollback mechanism can be applied to the other container groups. This ensures that services in container groups originally associated with the faulty physical acceleration device can continue to operate normally, while minimizing CPU resource consumption.

[0040] For example, in Figure 1 In the example, assume that the idle virtual devices 140-7 and 140-8 are only sufficient to handle the workload of one container group. Host 110 can assign a container group to one of container groups 130-1 and 130-2, and apply a CPU rollback mechanism to the other container group.

[0041] In some embodiments, host 110 may determine at least one container group based on the type of workload of each of the one or more container groups, and allocate a detected set of idle virtual devices to that at least one container group. In some such embodiments, host 110 may determine a priority order for allocating idle virtual devices to the one or more container groups based on the type of workload of each of the one or more container groups. Furthermore, host 110 may determine at least one container group based on the determined priority order and the number of idle virtual devices.

[0042] For example, during execution, host 110 can allocate a required number of virtual devices (e.g., determined in real time based on configuration or workload) to the one or more container groups in priority order until all idle virtual devices have been allocated, thereby allocating idle virtual devices to at least one container group.

[0043] The following will combine Figure 3 and Figure 4 To describe the allocation of idle virtual devices in more detail. It should be understood that... Figure 3 and Figure 4 The number of various components and the number of other types of components associated with a component are shown only as examples. Embodiments of this disclosure are applicable to any suitable number of physical acceleration devices, virtual devices, and / or container groups. In different embodiments, any suitable number of virtual devices can be created on a physical acceleration device, and any suitable number of virtual devices can be allocated to a container group.

[0044] Now for reference Figure 3 It illustrates an example scenario 300 where a physics acceleration device malfunctions. Example scenario 300 could, for example, occur in... Figure 1 On host 110. Example scenario 300 shows physical acceleration devices 320-1 and 320-2, where each physical acceleration device is mapped to multiple virtual devices. These virtual devices are assigned to three running container groups 330-1, 330-2, and 330-3. Container group 330-1 is assigned virtual devices 340-1 and 340-2 created on physical acceleration device 320-1, container group 330-2 is assigned virtual devices 340-3 and 340-4 created on physical acceleration device 320-1, and container group 330-3 is assigned virtual devices 340-5 and 340-6 created on physical acceleration device 320-2.

[0045] like Figure 3 As shown, in example scenario 300, the physics acceleration device 320-1 becomes unavailable, for example, due to a failure, thus making the associated virtual devices 340-1, 340-2, 340-3, and 340-4 unavailable. Consequently, container groups 330-1 and 330-2 will not have any available virtual devices to perform their workloads.

[0046] To ensure that the workloads of container groups 330-1 and 330-2 can continue to be processed, the host machine hosting the container groups (e.g., host 110) can (e.g., using a control plugin) detect whether other available virtual devices exist on other physical acceleration devices. Figure 3 As shown, the host can detect that there are still idle virtual devices 340-7 and 340-8 on the physical acceleration device 320-2. Therefore, before applying the CPU rollback mechanism, the host can allocate virtual devices 340-7 and 340-8 to container groups 330-1 and 330-2 to handle the workload of the container groups.

[0047] For illustrative purposes, in example scenario 300, assuming that each container group requires at least two virtual devices to execute the host workload when using the resources of physically accelerated devices, virtual devices 340-7 and 340-8 are insufficient to be allocated to the two container groups 330-1 and 330-2. In this case, the host can allocate virtual devices 340-7 and 340-8 to one of the container groups. For the remaining container groups that need to be patched, CPU rollback can be applied to reassign the workload of the remaining container groups to the host's CPU for processing.

[0048] Now for reference Figure 4 This illustrates example scenario 400 after fault handling of the container group in example scenario 300. For example... Figure 4 As shown, the host can allocate the detected idle virtual devices 340-7 and 340-8 to container group 330-2. For the remaining faulty container group 330-1, the host reassigns its workload to its CPU 401. In this way, the workloads of all container groups can continue to be processed normally, thus avoiding business interruption.

[0049] In some embodiments, the host may randomly select container groups from the container groups to be repaired to allocate virtual devices based on the number of idle virtual devices. In other embodiments, the host may consider Quality of Service (QoS) when selecting container groups, taking into account the workload type of each container group.

[0050] Different types of business services account for different proportions of the workload in a container group. For example, for businesses supported by QAT devices, recovery services require more resources for decompression, while ingestion services need to strike a balance between encryption (e.g., SHA1) computation and compression / decompression. On the other hand, different types of workloads consume different amounts of CPU resources. For example, when using CPU rollback, compression consumes more CPU resources than decompression, and decompression consumes more CPU resources than encryption.

[0051] Compared to container groups with workloads primarily focused on compression, applying CPU bounce to container groups with a higher proportion of encryption workloads has a smaller impact on the host CPU. Therefore, when allocating idle virtual devices, they should be prioritized for container groups with workloads primarily focused on compression / decompression. Take example scenario 400. If container group 330-1's workload is primarily SHA1, while 330-2's workload is primarily compression, then the host can allocate virtual devices 340-7 and 340-8 to container group 330-2, while using CPU bounce for container group 330-1.

[0052] In practice, the workload of a single container group often has multiple types. For example, the workload of container group 330-1 might consist of partial decompression and partial SHA1. In some embodiments, for a specific container group within one or more container groups requiring repair, the host can determine a priority score for that specific container group based on the proportion of each type of workload in its total workload and the computational resource consumption level of the respective type of workload. Based on the priority scores of the virtual devices in that group, the host can determine the priority order for allocating virtual devices to them.

[0053] In some embodiments, weights can be set for a specific type of workload based on its CPU consumption (e.g., CPU consumption per unit workload). These weights can depend on factors such as hardware conditions and can be predetermined based on experience, experimental testing, etc. For example, the host can store configuration files including weights for each type for use in fault handling. Alternatively, the host can track its operations during container group operation to determine the types of workloads within the container group and the proportion of each type in the total workload. Other appropriate methods for obtaining weight information and the proportion of different types of workloads within the container group can also be used.

[0054] Table 1 shows example weights for some example workload types in an example implementation. It should be understood that the specific numbers in Table 1 are merely examples and do not limit the scope of this disclosure. Depending on the implementation, other weight values ​​may also be used. Based on this, the host can determine the priority score of the container group according to the workload type of the container group and the CPU consumption weight of each type of workload. Furthermore, the host can sort the priority scores of the various container groups to determine the priority order of the container groups that need to be repaired. The higher the priority score of a container group, the higher the priority of the virtual device it is assigned to.

[0055] Table 1: Weights of Workload Types

[0056] CPU consumption weight SHA1 1 Unzip 2 compression 3

[0057] In one example implementation, suppose the workload of a container group consists of m% SHA1 type workloads and n% decompression type workloads. Then the priority score P of the container group can be expressed as follows, where cpu_weight SHA1 The CPU consumption weight for the SHA1 operation is indicated by cpu_weight. decomp Indicates CPU consumption weight for decompression operations

[0058] P = CPU weight SHA1 ×m%+cpu_weight decomp×n%

[0059] Taking the values ​​in Table 1 as an example, if container group 330-1 includes 80% of the SHA1 workload and 20% of the decompression workload, then its priority score can be determined as 1*80% + 2*20% = 1.2. If container group 330-2 includes 40% of the SHA1 workload and 60% of the decompression workload, then its priority score can be determined as 1*40% + 2*60% = 1.6. Therefore, the host can assign virtual devices 340-7 and 340-8 to container group 330-2, which has the higher score.

[0060] The above method also applies to situations where there are more container groups that need repair. The host can allocate the required number of virtual devices to each container group sequentially, based on their priority scores, until all available virtual devices have been allocated. If there are still remaining container groups without available virtual devices, the host will assign the workload of those remaining container groups to the CPU. In this way, the pressure on the host's CPU can be minimized when handling container group failures.

[0061] Once a failed physical acceleration device is restored, the VF devices on it become available again. In some embodiments, if the restoration of an unavailable VF device is detected, the host can detect whether there are container groups on the host that do not have available VF devices. That is, the VF devices currently allocated to the container groups are unavailable, and their workloads may need to be redirected to other components or are being processed by the host's CPU.

[0062] If such a container group is detected, the host can allocate the recovered VF devices to the detected container group for executing workloads targeting that container group / groups. In this way, the recovered accelerated computing resources can be reassigned to the container groups that need them, avoiding or stopping the application of CPU bounce mechanisms to that container group, thereby reducing CPU load.

[0063] If the VF device allocation takes load balancing into account during normal system operation, the number of non-idle VF devices on each physical acceleration device will be relatively even. When some PF devices fail, and after VF devices are redistributed to the container group according to embodiments of this disclosure, the affected workload will actually be transferred to the remaining available PF devices for processing. Thus, if the devices after the failed physical acceleration device do not undergo resource allocation adjustments after recovery, the load on the PF will become unbalanced.

[0064] Therefore, in some embodiments, after the restored VF devices are allocated to the aforementioned container groups, the host can detect the occupancy rate of the VF devices for each of the multiple physical acceleration devices on it. Based on this occupancy rate, the host can adjust the allocation of the VF devices of the multiple physical acceleration devices among the various container groups on the host, i.e., perform load balancing on these physical acceleration devices.

[0065] As an example, Figure 5 A schematic diagram 500 illustrates the resource usage of physical acceleration devices on an example system. Diagram 500 shows two normally functioning physical acceleration devices 520-1 and 520-2 in this example system. Furthermore, reference numerals 540-1 and 540-2 indicate the occupied virtual devices of physical acceleration devices 520-1 and 520-2 allocated to the container group, respectively. Assume that in this example, physical acceleration devices 520-1 and 520-2 have similar amounts of computing resources. Reference numerals 505-1 and 505-2 indicate the virtual device usage of these two physical acceleration devices at this time. As shown, the resource utilization of the two physical acceleration devices is approximately equal at this point.

[0066] and then, Figure 6 It shows Figure 5 The diagram 600 illustrates the resource usage of a physical acceleration device in the example system after a failure. Figure 6 As shown, a physics acceleration device 520-1 malfunctions, rendering the virtual devices on it unusable, as indicated by reference numeral 605-1. In this situation, according to the above embodiments of this disclosure, the host can detect that there are still idle virtual devices in the physics acceleration device 520-2. Therefore, the host can allocate these idle virtual devices to container groups whose workloads were originally handled by the physics acceleration device 520-1.

[0067] As described above, the host can determine the priority of these container groups and allocate the idle virtual devices of physical acceleration device 520-2 to these container groups in sequence. In this example, all the idle virtual devices on acceleration device 520-1 are allocated, so that the proportion of occupied virtual devices on it reaches 100%, as shown by reference numerals 640-2 and 605-2 in the attached figures.

[0068] In this example, there are still container groups on the host that do not have available virtual devices due to the failure of the physical acceleration device 520-1. The host can then assign the workload of these container groups to its CPU 601, as shown by reference numeral 640-1 in the attached figure.

[0069] Later, the physics acceleration device 520-1 can be restored during troubleshooting. Figure 7 It shows in Figure 6The example system illustrates resource usage after a failed physical acceleration device 520-1 is restored. After the physical acceleration device 520-1 is restored, the virtual devices on it become available again. At this point, the host can detect whether there are container groups without available virtual devices. In this example, the host can detect container groups whose workloads were previously assigned to CPU 601 for execution.

[0070] Furthermore, the host can allocate the available virtual devices on physical acceleration device 520-1 to the detected container groups to handle the workload of these container groups, as shown in reference numeral 740-1. In this case, all virtual devices on physical acceleration device 520-2 are still fully occupied, while the virtual device occupancy rate of physical acceleration device 520-1 is lower, as shown in reference numerals 705-1 and 705-2.

[0071] Therefore, in some embodiments, the host can check the virtual device utilization rate on each physical acceleration device and reallocate virtual devices based on this utilization rate, so that some container workloads can migrate from physical acceleration devices with high virtual device utilization rates to those with low virtual device utilization rates. This makes the resource usage of each physical acceleration device more balanced, thereby enabling more rational and efficient utilization of system resources. For example, for... Figure 7 For example, the host can perform load balancing on physics acceleration devices 520-1 and 520-2, thereby restoring the occupancy of each physics acceleration device to a state similar to... Figure 5 The state shown.

[0072] It should be understood that Figures 5 to 7 The number of devices shown is for illustrative purposes only and not a limitation. For example, load balancing can be similarly performed for more than two physical acceleration devices to bring the resource utilization of each physical device closer together. In the implementation, any appropriate load balancing method can be used to determine the specific reallocation of virtual devices.

[0073] Figure 8 A schematic block diagram of a device 800 that can be used to implement embodiments of the present disclosure is shown. Device 800 may be the device or apparatus described in the embodiments of the present disclosure. Figure 8As shown, device 800 includes a CPU 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of device 800. The CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804. Although not shown in... Figure 8 As shown, device 800 may also include a coprocessor.

[0074] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The various methods or processes described above can be executed by processing unit 801. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more steps or actions in the methods or processes described above can be performed.

[0076] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0077] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0078] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0079] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0080] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0083] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for scheduling container resources, comprising: In response to the fact that no virtual device is available in one or more container groups assigned to the host, it is detected whether there is a set of idle virtual devices that have not been assigned to the one or more container groups, wherein each virtual device is run by a physical acceleration device among a plurality of physical acceleration devices of the host; as well as In response to the absence of the set of virtual devices, the host's central processing unit is used to handle the workload of the one or more container groups.

2. The method according to claim 1, further comprising: In response to the presence of the set of virtual devices, the set of virtual devices is assigned to at least one of the one or more container groups to handle the workload of the at least one container group.

3. The method according to claim 2, further comprising: The central processing unit is used to process the workload of other container groups in the one or more container groups besides the at least one container group.

4. The method of claim 2, wherein assigning the set of virtual devices to the at least one container group comprises: The at least one container group is determined based on the type of workload of each of the one or more container groups.

5. The method of claim 4, wherein determining the at least one group of containers comprises: Based on the type of workload of each of the one or more container groups, determine the priority order for allocating the set of virtual devices to the one or more container groups; as well as The at least one container group is determined based on the priority order and the number of virtual devices in the group.

6. The method of claim 5, wherein determining the priority order for allocating the set of virtual devices to the one or more container groups comprises: For a given container group within the one or more container groups, a priority score is determined based on the proportion of each type of workload in the total workload of that container group and the level of computing resource consumption of the respective type of workload. as well as The priority order is determined based on the priority scores of each of the one or more container groups.

7. The method according to claim 1, further comprising: In response to the detection that an unavailable virtual device has been restored, detect whether there is a container group on the host that does not have an available virtual device; as well as In response to the detection of the container group, the restored virtual device is allocated to the container group for performing workloads targeting the container group.

8. The method according to claim 7, further comprising: After the restored virtual device is assigned to the container group, the occupancy rate of the virtual devices of each of the plurality of physical acceleration devices is detected; as well as Based on the occupancy rate, the allocation of virtual devices of the multiple physical acceleration devices in each container group on the host is adjusted.

9. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions including: In response to the fact that no virtual device is available in one or more container groups assigned to the host, it is detected whether there is a set of idle virtual devices that have not been assigned to the one or more container groups, wherein each virtual device is run by a physical acceleration device among a plurality of physical acceleration devices of the host; as well as In response to the absence of the set of virtual devices, the host's central processing unit is used to handle the workload of the one or more container groups.

10. The device according to claim 9, wherein the action further includes: In response to the presence of the set of virtual devices, the set of virtual devices is assigned to at least one of the one or more container groups to handle the workload of the at least one container group.

11. The device according to claim 10, wherein the action further comprises: The central processing unit is used to process the workload of other container groups in the one or more container groups besides the at least one container group.

12. The device of claim 10, wherein assigning the set of virtual devices to the at least one container group comprises: The at least one container group is determined based on the type of workload of each of the one or more container groups.

13. The apparatus of claim 12, wherein determining the at least one group of containers comprises: Based on the type of workload of each of the one or more container groups, determine the priority order for allocating the set of virtual devices to the one or more container groups; as well as The at least one container group is determined based on the priority order and the number of virtual devices in the group.

14. The device of claim 13, wherein determining the priority order for allocating the set of virtual devices to the one or more container groups comprises: For a given container group within the one or more container groups, a priority score is determined based on the proportion of each type of workload in the total workload of that container group and the level of computing resource consumption of the respective type of workload. as well as The priority order is determined based on the priority scores of each of the one or more container groups.

15. The device according to claim 9, wherein the action further comprises: In response to the detection that an unavailable virtual device has been restored, detect whether there is a container group on the host that does not have an available virtual device; as well as In response to the detection of the container group, the restored virtual device is allocated to the container group for performing workloads targeting the container group.

16. The device of claim 15, wherein the action further comprises: After the restored virtual device is assigned to the container group, the occupancy rate of the virtual devices of each of the plurality of physical acceleration devices is detected; as well as Based on the occupancy rate, the allocation of virtual devices of the multiple physical acceleration devices in each container group on the host is adjusted.

17. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform actions, the actions comprising: In response to the absence of available virtual devices among the virtual devices allocated to one or more container groups on the host, it is detected whether there exists a set of idle virtual devices not allocated to the one or more container groups, wherein each virtual device is run by a physical acceleration device among a plurality of physical acceleration devices on the host; and In response to the absence of the set of virtual devices, the host's central processing unit is used to handle the workload of the one or more container groups.

18. The computer program product of claim 17, wherein the action further comprises: In response to the presence of the set of virtual devices, the set of virtual devices is assigned to at least one of the one or more container groups to handle the workload of the at least one container group.

19. The computer program product of claim 18, wherein the action further comprises: The central processing unit is used to process the workload of other container groups in the one or more container groups besides the at least one container group.

20. The computer program product of claim 18, wherein assigning the set of virtual devices to the at least one container group comprises: The at least one container group is determined based on the type of workload of each of the one or more container groups.

Images