A resource allocation method and device, electronic equipment and storage medium

Abstract

The application discloses a resource allocation method applied to the technical field of computers, and comprises the following steps: determining target GPU driver allocation information, wherein the target GPU driver allocation information is used for indicating the demand situation of different GPU drivers; acquiring the currently loaded driver of each GPU device; determining the target GPU device used for driver conversion and the target GPU driver converted from the target GPU device according to the target GPU driver allocation information and the currently loaded driver of each GPU device; and converting the currently loaded driver of the target GPU device into the target GPU driver. Through the method, the drivers loaded by the multiple GPU devices in the same electronic device can be automatically switched on demand, so that the GPU resources in one electronic device can be flexibly allocated on demand, and the GPU resource utilization efficiency in a single electronic device is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a resource allocation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In cluster technologies such as Kubernetes (K8S), GPU (Graphics Processing Unit) resources can be used on nodes either as containers or as virtual machines. GPU resources refer to the computing power of the GPU. Furthermore, a cluster has multiple nodes, each with multiple GPU devices. Each GPU device requires loading the corresponding GPU device driver for the container or virtual machine to be used. Consequently, managing GPU resources in a cluster is complex. Therefore, how to rationally allocate GPU resources within a cluster is crucial for its GPU utilization efficiency.

[0003] Existing cluster technologies focus solely on configuring GPU resources on a node-by-node basis. Specifically, current technologies configure only one driver for a single node, ensuring that all GPU devices on that node run the same GPU driver type, thereby utilizing all GPU resources on a single node in a single manner, such as containers or virtual machines.

[0004] However, using all GPU devices on a single node through containers or virtual machines can easily lead to idle GPU devices on that node due to insufficient or terminated business operations, resulting in wasted GPU resources. Therefore, improving GPU resource utilization efficiency for individual nodes has become a new requirement. Summary of the Invention

[0005] This application provides a resource allocation method, apparatus, electronic device, and storage medium that can enable multiple GPU devices within a single node to load different drivers and automatically switch driver types as needed, thereby flexibly allocating GPU resources within a single node to improve GPU resource utilization efficiency.

[0006] The first aspect of this application provides a resource allocation method applied to an electronic device, which includes multiple GPU devices, each GPU device having at least two different GPU drivers installed. The method includes: Determine the target GPU resource allocation information, which includes the target GPU driver allocation information; Get the drivers currently loaded for each GPU device; Based on the target GPU driver allocation information and the drivers currently loaded on each GPU device, determine the target GPU device for driver conversion from multiple GPU devices, as well as the target GPU driver converted from the target GPU device. Convert the driver currently loaded on the target GPU device to the target GPU driver.

[0007] The second aspect of this embodiment provides a resource allocation device applied to an electronic device, the electronic device including multiple GPU devices, each GPU device having at least two different GPU drivers installed, the device comprising: The determination unit is used to determine the target GPU driver allocation information, which is used to indicate the requirements of different GPU drivers. The acquisition unit is used to acquire the drivers currently loaded for each GPU device; The determining unit is also used to determine, from multiple GPU devices, the target GPU device for driver conversion and the target GPU driver converted from the target GPU device, based on the target GPU driver allocation information and the driver currently loaded on each GPU device; The conversion unit is used to convert the driver currently loaded on the target GPU device into the target GPU driver.

[0008] The third aspect of this embodiment provides an electronic device that includes multiple GPU devices, each GPU device having at least two different GPU drivers installed. The electronic device includes: Memory and processor, memory and processor coupled; Memory is used to store one or more computer instructions; The processor is used to execute one or more computer instructions to implement the method provided in the first aspect above.

[0009] The fourth aspect of this embodiment provides a computer-readable storage medium storing one or more computer instructions applied to an electronic device, the electronic device including multiple GPU devices, each GPU device having at least two different GPU drivers installed, the computer instructions being executed by a processor to implement the method provided in the first aspect above.

[0010] The technical solution provided in this application first determines the target GPU driver allocation information, which indicates the demand for different GPU drivers. Then, it obtains the drivers currently loaded on each GPU device. Next, based on the target GPU driver allocation information and the drivers currently loaded on each GPU device, it determines the target GPU device for driver conversion and the target GPU driver converted from the multiple GPU devices. Finally, it converts the driver currently loaded on the target GPU device into the target GPU driver.

[0011] This resource allocation method provides nodes with the ability to automatically switch GPU device drivers on demand. When the demand for GPU resource allocation within a node changes, the node automatically adjusts the current GPU device drivers based on the determined target GPU resource allocation information, ensuring that the GPU resource allocation matches the preset allocation. This method allows for flexible switching of GPU device drivers within a single node, enabling GPU resources to be used flexibly in the form of containers or virtual machines. This avoids situations where a large amount of GPU resources are idle within a single node, thus improving GPU resource utilization efficiency. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the resource allocation method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the method for determining a target GPU device based on the idle state of a GPU device according to an embodiment of this application. Figure 3 This is a schematic diagram illustrating the application of GPU resources on a K8S node in an embodiment of this application; Figure 4 This is a diagram illustrating the allocation of GPU resources in a K8S node; Figure 5 This is a schematic diagram of the structure of a resource allocation device provided in an embodiment of this application; Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions of this application, the application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. However, this application can be implemented in many other ways different from those described above. Therefore, based on the embodiments provided in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0014] It should be noted that the terms "first," "second," "third," etc., in the claims, specification, and drawings of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. Such data are interchangeable where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown or described herein. Furthermore, the terms "comprising," "having," and their variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0015] In cluster technologies like Kubernetes, multiple servers are used simultaneously within a cluster, each referred to as a node. The cluster's vast computing resources provide the runtime environment for numerous applications. GPU resources, as a type of computing resource, are essential for the operation of GPU applications. GPU applications can run in containers or virtual machines. For these two different application environments, the GPU device needs to load the corresponding GPU device driver for the container or virtual machine. This ensures that the GPU resources on the GPU device are compatible with the application environment of the container or virtual machine, allowing the GPU resources to be allocated and used.

[0016] Because a cluster has multiple nodes, and each node has multiple GPU devices, configuring GPU resources requires loading different drivers, which complicates GPU resource management within the cluster. Current technologies, to facilitate GPU resource allocation, focus on configuring GPU resources on a node-by-node basis. This means configuring a single driver for all GPU devices within a single node, allowing all GPU resources on a single node to be used in a single manner, such as containers or virtual machines. Overall GPU resources are managed by managing the different driver types on different nodes.

[0017] However, when using the above method, the GPU resources in a node are only allocated to one type of GPU application. For the same node, there may be situations where the GPU resources required by the running application are far less than the node's total GPU resources, resulting in insufficient resources, or the running application may terminate, leading to the idle GPU devices on a single node, wasted GPU resources, and low utilization. Therefore, improving the GPU resource utilization efficiency for individual nodes has become a new requirement.

[0018] To address the aforementioned needs, this application provides a resource allocation method, apparatus, electronic device, and storage medium. The method automatically converts the driver type of some GPU devices based on target GPU driver allocation information indicating GPU driver demand, ensuring that the number of target GPU drivers loaded on the GPU device matches the target driver allocation information. This allows for flexible on-demand allocation of GPU resources on a single device, enabling efficient utilization of GPU resources. The following detailed description, in conjunction with specific embodiments and accompanying drawings, further illustrates the method, apparatus, electronic device, and computer-readable medium of this application.

[0019] The following combination Figure 1 The resource allocation method provided in this application will be described in detail. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer-executable instructions, and in some cases, the steps shown can be executed in a logical order different from that shown in the flowchart.

[0020] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the resource allocation method provided in this application. The resource allocation method includes steps S101-S104, as detailed below.

[0021] First, it should be noted that this method can be applied to multiple node devices in cluster technologies such as K8S, or to a specific device in the cluster, or only to a single electronic device; there are no restrictions on this.

[0022] This method is applied to electronic devices that include multiple GPU devices, each with at least two different GPU drivers installed. The conversion process involves switching between installed GPU drivers, meaning the target GPU driver is already installed on the GPU device. Steps S101-S104 can be executed at any time when the required GPU driver is installed on each GPU device in the electronic device, initiating the first step of on-demand GPU driver conversion.

[0023] S101. Determine the target GPU driver allocation information.

[0024] First, this step requires determining the target GPU driver allocation information. This GPU driver allocation information indicates the different drivers loaded on different devices, specifically the types of GPU drivers and the number of GPUs corresponding to each driver. The target GPU driver allocation information indicates the demand for different GPU drivers, meaning the target GPU driver allocation information needs to be converted into the appropriate GPU driver type and quantity.

[0025] For example, an electronic device has eight GPU drivers, each with driver A and driver B installed. If a user wants to configure these GPU drivers so that three devices load driver A and five devices load driver B, then the corresponding target GPU driver allocation information is as follows: three GPU devices load driver A and five devices load driver B.

[0026] The target GPU driver allocation information includes at least the target GPU driver and its corresponding quantity relationship. The target GPU driver is the type of GPU driver that needs to be converted, and the corresponding quantity relationship indicates the required number of GPU devices to load the target GPU driver.

[0027] Specifically, the target GPU driver allocation information includes the quantity of target GPU drivers, or the ratio of target GPU drivers. The quantity of target GPU drivers refers to the specific number of target GPU drivers and various types of target GPU drivers. For example, the target GPU driver allocation information indicates that there are 3 GPU devices loading driver A and 5 GPU devices loading driver B. The ratio of target GPU drivers refers to the proportion of target GPU drivers and various types of target GPU drivers. For example, if the ratio of driver A to driver B in the target GPU driver allocation information is 1:1, then the specific number of target GPU drivers will vary accordingly based on the total number of GPU devices. For example, if there are a total of 8 GPU devices, then there will be 4 devices loading driver A and 4 devices loading driver B. If there are a total of 16 GPU devices, then there will be 8 devices loading driver A and 8 devices loading driver B.

[0028] After determining the target GPU driver allocation information, it is also necessary to obtain the driver currently loaded for each GPU device, as detailed in step S102.

[0029] S102. Obtain the currently loaded drivers for each GPU device.

[0030] Obtaining the currently loaded drivers for each GPU device refers to retrieving the driver types currently loaded for each GPU device. It's important to understand that while retrieving the currently loaded drivers for each GPU device, information about the GPU device itself is also retrieved. Furthermore, the number of various GPU devices loaded can be statistically analyzed. Therefore, the purpose of this step is essentially to obtain current GPU driver allocation information, and thus it is not limited to simply retrieving the types of drivers currently loaded for each GPU device; it should also be considered as simultaneously retrieving GPU device information and being able to summarize and statistically analyze various pieces of information.

[0031] There are several methods for obtaining the currently loaded drivers for each GPU device. For example, one approach is to first check the current information for each GPU device to see which drivers are currently loaded, then categorize each GPU device to obtain the current GPU driver allocation information. Alternatively, each GPU driver can be used as an index to iterate through all GPU devices, obtaining the number of GPU devices loading each type of driver, as well as information about each GPU device loading each type of driver. Furthermore, there are other specific methods for obtaining the currently loaded drivers for each GPU device, which are not specifically limited here.

[0032] After obtaining the driver currently loaded for each GPU device, step S103 is executed to determine the target GPU device for driver conversion and the target GPU driver converted from the target GPU device.

[0033] S103. Based on the target GPU driver allocation information and the drivers currently loaded on each GPU device, determine the target GPU device for driver conversion from multiple GPU devices, as well as the target GPU driver converted from the target GPU device.

[0034] Once the target GPU driver allocation information and the GPU drivers currently loaded on each GPU device are obtained, the required GPU drivers to be converted, the target GPU drivers to be converted into, and the specific quantity required can be determined based on the current driver allocation situation to meet the target GPU driver allocation information. After determining the target driver and the required quantity, the target GPU device for driver conversion and the target GPU driver to be converted from the multiple GPU devices are selected.

[0035] The process involves determining the target GPU device for driver conversion and the target GPU driver to be converted from that device. Since the target GPU driver will inevitably increase after the conversion, the specific steps are as follows: The target GPU driver allocation information is compared with the drivers currently loaded on each GPU device to determine the target GPU driver to be added and the required quantity. Based on the target GPU driver to be added and the required quantity, the target GPU device for driver conversion and the target GPU driver to be converted from each GPU device are determined. It's important to understand that if the target GPU device for driver conversion is determined, the converted GPU driver is also specified.

[0036] For example, there are a total of 8 GPU devices. Step S102 reveals that there are currently 3 devices loading driver A (a, b, and c) and 5 devices loading driver B (d, e, f, g, and h). The target GPU driver allocation information indicates that the ratio of devices loading driver A to those loading driver B is 1:1. Therefore, a device loading driver B needs to be selected from ah and converted to load driver A. At this point, driver B is the GPU driver to be converted, driver A is the target GPU driver, and the target GPU device is the device selected from dh for converting the GPU driver.

[0037] It's important to understand that the purpose of comparing the target GPU driver allocation information with the drivers currently loaded on each GPU device is to obtain discrepancies and, based on these discrepancies, to align the current GPU driver allocation with that indicated by the target GPU driver allocation information. The process of satisfying the target GPU driver allocation information can either be done completely or gradually.

[0038] Specifically, fully satisfying the target GPU driver allocation information means that a certain process is used to determine enough target GPU devices for driver conversion, as well as the target GPU devices converted into target GPU drivers, so that a single driver conversion can satisfy the target GPU driver allocation information.

[0039] Gradually satisfying the target GPU driver allocation information means that the target GPU device for driver conversion and the target GPU driver converted from the target GPU device can be determined through multiple deterministic processes. In other words, the driver of some GPU devices can be converted multiple times until the target GPU driver allocation information is satisfied.

[0040] The process of gradually satisfying the target GPU driver allocation information can be a process of gradually reducing the differences. Specifically, this involves determining the differences between the currently running driver of each GPU device and the target GPU driver allocation information. To reduce these differences, the target GPU device for driver conversion and the target GPU driver converted from the multiple GPU devices are identified.

[0041] For example, there are a total of 8 GPU devices, with 4 currently loaded for driver A and 4 for driver B. The target GPU driver allocation information is as follows: 6 GPU devices are needed to load GPU driver A, and 2 GPU devices are needed to load driver B. To fully satisfy the target GPU driver allocation information, 2 target GPU devices loaded with GPU driver B need to be identified when determining the target GPU devices, so that enough GPU devices can be converted at once.

[0042] To progressively satisfy the target GPU driver allocation information, we can first identify a target GPU device loaded with GPU driver B, convert it to load driver A, and then select one of the remaining three GPU devices loaded with GPU driver B for driver conversion. Furthermore, during this process, we can initially determine that there are two fewer devices loaded with GPU driver A and two more devices loaded with GPU driver B. Therefore, to reduce the discrepancy, we need to convert the two devices loaded with GPU driver B to load GPU driver A.

[0043] In determining the target GPU device for driver conversion and the target GPU driver converted from the target GPU device, the target GPU driver allocation information can be compared with the current GPU driver allocation information obtained based on the GPU drivers currently loaded on each GPU device. This comparison yields the difference between the demand for various GPU drivers and the number of devices currently loading various GPU drivers. The target GPU device and the target GPU driver converted from the target GPU device are then determined based on this difference. Alternatively, the current quantity of each GPU device can be directly calculated by subtracting the demand for each GPU driver from the target GPU driver allocation information. The target GPU device and the target GPU driver converted from the target GPU device are then determined based on the difference. This difference is essentially also difference information. Other methods for obtaining difference information are also applicable to this embodiment and are not limited here.

[0044] Besides the method described above for determining the target GPU driver to be added and the required number of target GPU drivers through difference information, other methods can also be used, such as setting conditional functions, module judgment, etc. For example, the total number of GPU devices is 8, and currently there are 4 devices each loaded with driver A and driver B. The target GPU driver allocation information is: 6 GPU devices are needed to load driver A, and 2 GPU devices are needed to load driver B. In this case, the GPU devices loaded with driver A are traversed and counted. After traversing all GPU devices, the count is 4, and the condition set according to the target GPU driver allocation information (A=6) is not triggered. Then, the devices loaded with driver B are converted to load driver A one by one. When 2 are successfully converted, the condition A=6 is triggered, indicating that the conversion is complete, and the counting and driver conversion stop. Alternatively, the GPU devices loaded with driver B are traversed and counted. When the count reaches the second GPU device loaded with driver B, the set condition (B=2) is triggered, the counting stops, and all other uncounted GPU devices loaded with driver B are converted to load driver A.

[0045] It is important to understand that the difference between this method and the method for obtaining difference information lies in the implementation of other methods for determining the target GPU driver to be added and the required amount of the target GPU driver. Specifically, they do not require a comparative calculation process to obtain difference values ​​to determine the required amount of the target GPU driver to be added. However, these other methods still fall under the same concept of reducing differences and should be included within the scope of the embodiments outlined in this application.

[0046] Once the target GPU device for driver conversion and the target GPU driver converted from the target GPU device are determined, step S104 can be executed to complete the conversion of the GPU driver loaded on the target GPU device, so that the allocation of GPU resources is consistent with the demand.

[0047] S104. Convert the driver currently loaded on the target GPU device into the target GPU driver.

[0048] Once the target GPU device and its corresponding target GPU driver are identified, the currently loaded driver on the target GPU is converted to the target GPU driver. Specifically, this involves either replacing the currently loaded GPU driver on the target GPU device with the target GPU driver, or reloading the GPU driver on the target GPU device to ensure it loads the target GPU driver, thus completing the GPU driver conversion.

[0049] Furthermore, it is important to understand that a GPU device may contain multiple sub-devices, and when converting the GPU driver, the GPU drivers for all sub-devices must be converted.

[0050] Furthermore, since GPU resources are allocated to GPU applications when they are used, continuously allocating GPU resources to GPU applications keeps the GPU devices in an allocable state. This can easily lead to driver conversion failures, preventing the successful replacement of the loaded driver. Therefore, in some implementations, the allocation of GPU resources for each GPU device should be paused before driver conversion, and the allocation function should be restarted after all driver conversions are completed. In other words, the allocation of GPU resources for each GPU device should be paused before converting the currently loaded driver of the target GPU device to the target GPU driver. After converting the currently loaded driver of the target GPU device to the target GPU driver, the function of allocating GPU resources for each GPU device should be restarted.

[0051] Through steps S101-S104, the GPU drivers loaded on multiple GPU devices can be converted as needed, so that the allocation of GPU drivers is consistent with the expected allocation, thereby enabling each GPU device to be used by the application corresponding to the GPU driver, thus achieving flexible allocation of GPU resources.

[0052] The above method is the basic method of the embodiments of this application. Based on the above method, corresponding changes can be made according to different scenario requirements. The following describes the above method in more detail with reference to specific scenarios.

[0053] In the above methods, there are many specific ways to determine the target GPU driver allocation information. This embodiment provides two methods: Method 1 is to determine the target GPU driver allocation information based on the user's setting instructions, and Method 2 is to determine the target GPU driver allocation information based on the GPU resource requirements of the application to be run. These two methods will be described in detail below.

[0054] Method 1: In response to driver setting instructions, determine the target GPU driver allocation information. The driver setting instructions include the requirements for different GPU drivers.

[0055] Users can configure the allocation of GPU drivers within electronic devices. Specifically, users can input the number or ratio of target GPU drivers through the human-computer interaction function on the electronic device, or they can configure it through other devices and send the configuration commands to the electronic device via the messaging function. The specific configuration method can be changed according to different human-computer interaction methods, and no specific restrictions are imposed here.

[0056] When an electronic device receives a driver configuration command, it responds to the command and retrieves the target GPU driver allocation information indicated by the command. Specifically, this involves obtaining the requirements for different GPU drivers from the command and using this information as the target GPU driver allocation information.

[0057] For example, an electronic device has a total of 8 GPU devices, with drivers A and B loaded respectively. When the user configures the GPU resources on the graphical user interface of the electronic device, they input A=2 and B=6. After receiving the user's driver configuration command, the electronic device determines the target GPU driver allocation information as follows: 2 GPU devices loaded with driver A and 6 GPU devices loaded with driver B. Alternatively, the user sends a driver configuration command of A=2 and B=6 to the target node M through a node management device. When the target node M recognizes this driver configuration command from the received information, it determines the target GPU driver allocation information accordingly.

[0058] Method 2: Determine the target GPU driver allocation information based on the GPU resource requirements of the application to be run and the current GPU resources.

[0059] This method determines the target GPU driver allocation information based on the GPU resource requirements of the application to be run. Specifically, in determining the target GPU driver allocation information, it is first necessary to determine whether the current GPU resources meet the GPU resource requirements of the application to be run, based on the GPU resource requirements of the application to be run and the current GPU resources, and then determine the target GPU driver allocation information accordingly.

[0060] For example, the GPUs in an electronic device are all of the same model, and there are two applications to run: application a and application b. Application a requires a GPU device with driver A to run, and requires 50% of the computing power of one GPU device to run normally. Application b requires a GPU device with driver B to run, and requires 30% of the computing power of one GPU device to run normally. Currently, there are three GPU devices with driver a, and only one of them has 20% of its computing power available. There are five GPU devices with driver b, and two of them have unused computing power. Therefore, it can be determined that the GPU resources of the GPU device with driver A are insufficient to meet the GPU resource requirements of the applications to run, while the GPU resources of the GPU device with driver B are sufficient. This leads to the determination of the target GPU driver allocation information to convert a GPU device with driver B to one with driver A.

[0061] Besides these two methods, other methods can be used to determine the target GPU driver allocation information, such as setting fixed automatic conversion rules through functions, such as setting a minimum threshold for the number of devices to load a certain driver, or setting a threshold for the minimum ratio of two certain drivers. The specific settings can be modified according to different requirements, and will not be elaborated further here.

[0062] Furthermore, in step S103 of the aforementioned basic method, when determining the target GPU device for driver conversion and the target GPU device converted into a target GPU driver from among the various GPU devices based on the target GPU driver to be added and the required quantity, different determination methods can be set according to different requirements. This embodiment provides a method for determining the target GPU device based on the idle state of the GPU device. Specific details... Figure 2 Provide a detailed explanation. Figure 2 This is a flowchart illustrating a method for determining a target GPU device based on its idle state. The specific steps are described in S201-S202 below.

[0063] S201. Determine at least one first alternative GPU device that is in an idle state from among multiple GPU devices.

[0064] First, determine the idle status of each GPU device. The idle status indicates that the GPU device is not occupied, or that the GPU device has not been assigned a GPU application.

[0065] If there are idle GPU devices, all idle GPU devices are identified as the first candidate GPU devices, and step S202 is executed. If there are no idle GPU devices, a message indicating insufficient idle devices is issued, or the first candidate GPU device is identified only after an idle GPU device becomes available, and step S202 is executed.

[0066] S202. Based on the target GPU driver to be added and the demand, determine the target GPU device for driver conversion and the target GPU driver converted from each of the first candidate GPU devices.

[0067] Once the first pool of candidate GPU devices is determined, the target GPU devices for driver conversion, and the target GPU drivers converted from those devices, can be selected from the pool of candidate GPU devices based on the target GPU drivers to be added and the required quantity. In other words, the target GPU devices are determined based on the type and quantity of the target GPU drivers.

[0068] Since the degree to which idle GPU devices meet the demand of the target GPU driver to be added varies, the methods for determining the target GPU devices also differ.

[0069] If the number of first-selection GPU devices is greater than or equal to the required number of target GPU drivers to be added, it means that the idle GPU devices are sufficient to meet the conversion requirements. In this case, the required number of GPU devices can be selected from the idle GPU devices for driver conversion. Specifically: if the number of first-selection GPU devices is greater than or equal to the required number of target GPU drivers to be added, the required number of GPU devices are selected from the first-selection GPU devices as the target GPU devices for driver conversion.

[0070] If the number of first-selection GPU devices is less than the demand for the target GPU driver, it means that the idle GPU devices cannot meet the conversion requirements. In this case, to reduce the discrepancy, all idle GPU devices can be converted first, and then converted again when new idle devices become available. Specifically: if the number of first-selection GPU devices is less than the demand for the target GPU driver, each first-selection GPU device is designated as the target GPU device. If a second-selection GPU device is detected that is idle, it is designated as the target GPU device, until the number of target GPU devices equals the demand for the target GPU driver.

[0071] In addition to determining the target GPU device based on its idle state, it can also be determined based on its usage. For example, if a GPU device has been allocated a GPU application, but that application, while consuming computing power, remains in a closed, inactive state for an extended period, the application on that GPU device can be removed, thus identifying it as the target GPU device. Besides these, there are several other feasible methods for determining the target GPU device, which will not be specifically limited here.

[0072] In addition, based on the methods described above, an automatic GPU resource allocation function can be configured for electronic devices. For example, if an electronic device has a total of 8 GPU devices, and all 4 GPU devices loading GPU driver A are occupied, while the 4 GPU devices loading GPU driver B are not used, it indicates that the GPU resource allocation is unreasonable, and there is a risk of insufficient devices loading GPU driver A. In this case, some or all of the GPU devices loading GPU driver B can be automatically converted to loading driver A.

[0073] The specific steps for converting all GPU devices loaded with GPU driver B to those loaded with driver A can be summarized as follows: Identify each first GPU device currently loading the first GPU driver, and each second GPU device currently loading the second GPU driver; when all second GPU devices are in a non-idle state, and there is an idle device among the first GPU devices, convert the GPU driver currently loaded on the idle first GPU device to the second GPU driver. Correspondingly, when converting only some GPU drivers, the number of devices to be converted can be reduced accordingly in the above-described method, depending on the specific approach.

[0074] The above embodiments provide a detailed description of the resource allocation method, which can be applied to different application scenarios. The following section elaborates on the specific implementation process of applying the above method to a K8S cluster environment. Details are as follows.

[0075] First, it's important to understand that this method applies to a single Kubernetes node, i.e., a single server. Furthermore, each Kubernetes node has multiple GPU devices, and each GPU device can run GPU applications within containers or virtual machines by loading GPU container drivers and GPU virtual machine drivers.

[0076] like Figure 3 As shown, Figure 3 This diagram illustrates the application of GPU resources on a Kubernetes node. For GPU container applications, the GPU device loads a driver from the host machine, such as the NVIDIA GPU Driver (referred to as the GPU container driver in this embodiment). This driver shares the host machine's libraries within the container environment, enabling the GPU resources to be used by the GPU container application. For GPU virtual machine applications, the GPU device loads a GPU virtual machine driver (VFIO-PCI) and connects directly to the virtual machine via PCI (Peripheral Component Interconnect) passthrough. The corresponding GPU driver is then installed within the virtual machine, allowing the GPU resources to be used by the GPU virtual machine application.

[0077] Therefore, on a single node of a Kubernetes cluster, it is necessary to install the GPU container driver (GPU Driver) and the GPU virtual machine driver (VFIO-PCI) for each GPU device. Specifically, the GPU Driver and VFIO-PCI Driver can be deployed on the host machine, and each GPU device can load them separately.

[0078] Furthermore, when using the above resource allocation method to manage and allocate GPU container resources and GPU virtual machine resources in a Kubernetes cluster, it is necessary to first configure the runtime environment for the normal use of containers and virtual machines, i.e., the initialization configuration process. Specifically, this involves installing GPU mounting components on the nodes to mount the GPU devices into the containers, configuring the container engine, and completing the container environment configuration, such as configuring Docker or containerd. Additionally, it involves enabling the IOMMU (Input / Output Memory Management Unit) function on the nodes and loading the vfio-pci kernel module to complete the virtual machine environment configuration. The IOMMU function is used to map virtual memory addresses to physical memory addresses.

[0079] When using the resource allocation method described above, a data volume ConfigMap can be created in the initial state. This data volume exists as a file and is used to store the target GPU driver allocation information. The ConfigMap can either contain only one data volume to ensure consistent GPU driver allocation across all nodes, or it can contain data volumes corresponding to each node to configure the GPU driver allocation information for each node separately.

[0080] Furthermore, a GPU driver conversion module, such as `gpu-driver-controller`, can be created. This GPU driver conversion module is a program module used to implement the resource allocation method provided in the basic embodiment above. The GPU driver conversion module can be deployed to each GPU node in the form of a daemon set. Specifically, when a new node is added, the daemon set adds a GPU driver conversion module to the new node. When a node is deleted, the GPU driver conversion module of the deleted node is reclaimed.

[0081] Simultaneously, the ConfigMap needs to be mounted to the GPU driver conversion module. More specifically, the ConfigMap should be mounted to the Pod, the basic unit of the GPU driver conversion module, so that the GPU driver conversion module can read the target GPU driver configuration information of the node. Furthermore, when the target GPU driver configuration information changes, the GPU driver conversion module can detect the change, thereby re-obtaining the current GPU configuration and performing GPU driver conversion according to the new GPU configuration. The following combines... Figure 4 The process of applying the above method through the GPU driver conversion module is described in detail, among which... Figure 4 This is a diagram illustrating the allocation of GPU resources in a K8S node.

[0082] First, the GPU driver conversion module determines the target GPU driver allocation information from the data volume.

[0083] like Figure 4 As shown, after the ConfigMap is mounted to the GPU driver conversion module, the GPU driver conversion module listens to the ConfigMap. The GPU driver conversion module determines the target GPU driver allocation information from the data volume in two ways: first, when the data volume is initially created and the target GPU driver allocation information is set initially; second, when the set target GPU driver allocation information is modified.

[0084] When the target GPU driver allocation information is initially set, after the ConfigMap is created, all GPU devices within the node load GPU drivers according to the initial GPU driver allocation information in the ConfigMap. For example, the initial GPU driver allocation information carried when the ConfigMap is created is: GPU container driver and GPU virtual machine driver are 1:1. Since the GPU devices in the initial state have not loaded any drivers, when configuring the drivers loaded by each GPU device according to the target GPU driver allocation information, half of the GPU devices load the GPU container driver, and the other half load the GPU virtual machine driver.

[0085] When the target GPU driver allocation information is changed, the files in the ConfigMap will be modified accordingly. At this time, the GPU driver conversion module will detect the ConfigMap change and determine the latest GPU driver allocation information as the target GPU driver allocation information.

[0086] After determining the target GPU driver allocation information from the data volume, the GPU driver conversion module will obtain the driver currently loaded for each GPU device.

[0087] Specifically, when the GPU driver conversion module detects a change in the ConfigMap, it indicates that the required GPU resources have changed. At this point, it obtains the GPU drivers loaded by each GPU device within the node to understand the current GPU driver loading status and prepares to determine the target GPU device for driver conversion.

[0088] Then, the GPU driver conversion module determines the target GPU device for driver conversion and the target GPU driver converted from multiple GPU devices based on the target GPU driver allocation information and the driver currently loaded on each GPU device.

[0089] For example, the GPU driver conversion module compares the target GPU driver allocation information with the current GPU driver loading status, using the difference as the comparison result. Then, based on the comparison result, it determines the target GPU device for driver conversion and the target GPU driver from idle GPU devices. Alternatively, the GPU driver conversion module can use other methods for determining the target GPU device described in the above embodiments, which will not be elaborated here.

[0090] Once the target GPU device and target GPU driver are identified, the GPU driver conversion module converts the driver currently loaded on the target GPU device into the target GPU driver. Furthermore, since a GPU device contains multiple sub-devices, all GPU sub-devices within the target GPU device must be converted before the conversion process can be completed.

[0091] Furthermore, within a node, allocating GPU resources essentially involves scheduling and allocating these resources to GPU applications. This scheduling and allocation is implemented by GPU resource management plugins, such as container GPU resource management plugins and virtual machine GPU resource management plugins. Therefore, to avoid tasks being assigned to different GPU devices during the GPU driver conversion process, which could affect the conversion results, the container and virtual machine GPU resource management plugins can be paused after the GPU driver conversion module detects a ConfigMap change. These plugins should then be restarted after the GPU conversion is complete to ensure that the available GPU resources on the node match the actual available GPU resources.

[0092] Furthermore, after the GPU driver conversion is complete on the target GPU device, the GPU resource management plugin will rediscover the GPU resources and report them. Specifically, the container GPU resource management plugin and the virtual machine GPU resource management plugin will respectively check the GPU resources corresponding to the GPU device loading the GPU container driver and the GPU device loading the GPU container driver, and count the total amount of GPU resources, thereby reporting the GPU resources available to the GPU container application and the GPU virtual machine application respectively. After the resource status of the container GPU resources and virtual machine GPU resources is reported, the container GPU resources and virtual machine GPU resources will be allocated to the GPU container application and the GPU virtual machine application respectively, so that the converted GPU resources can be fully utilized.

[0093] Figure 5 This is a schematic diagram of a resource allocation device provided in an embodiment of this application. The following is in conjunction with... Figure 5 The embodiments of this application will be described in detail below. It should be understood that this resource allocation device is used to implement the various methods provided in the above method embodiments. The embodiments described below are used to explain the technical solutions of this application and are not intended to limit actual use.

[0094] like Figure 5 As shown in the provided resource allocation device 500, the device is applied to an electronic device that includes multiple GPU devices, each GPU device having at least two different GPU drivers installed. The device includes: The determining unit 501 is used to determine the target GPU driver allocation information, which is used to indicate the demand of different GPU drivers. The acquisition unit 502 is used to acquire the driver currently loaded for each GPU device; The determining unit 501 is further configured to determine, from multiple GPU devices, the target GPU device for driver conversion and the target GPU driver converted from the target GPU device, based on the target GPU driver allocation information and the driver currently loaded on each GPU device. The conversion unit 503 is used to convert the driver currently loaded on the target GPU device into the target GPU driver.

[0095] In one feasible implementation, the determining unit 501 is further configured to compare the target GPU driver allocation information with the drivers currently loaded on each GPU device to determine the target GPU driver to be added and the required amount of the target GPU driver. Based on the target GPU driver to be added and the demand, determine the target GPU device for driver conversion from each GPU device, as well as the target GPU driver converted from the target GPU device.

[0096] It is also used to determine at least one first alternative GPU device that is in an idle state from a plurality of GPU devices; Based on the target GPU driver to be added and the demand, the target GPU device for driver conversion and the target GPU driver converted from each of the first candidate GPU devices are determined.

[0097] It is also specifically used for, If the number of first candidate GPU devices is greater than or equal to the demand for the target GPU driver to be added, select the GPU device with the demand from the first candidate GPU devices as the target GPU device for driver conversion.

[0098] If the number of first candidate GPU devices is less than the demand for the target GPU driver to be added, each first candidate GPU device is determined as the target GPU device, and if a second candidate GPU device in an idle state is detected, the second candidate GPU device is determined as the target GPU device, until the number of target GPU devices is equal to the demand for the target GPU driver to be added.

[0099] The determining unit 501 is also used to determine the target GPU driver allocation information in response to the driver setting instruction, wherein the driver setting instruction includes the requirements for different GPU drivers; or, Used to determine the target GPU driver allocation information based on the GPU resource requirements of the application to be run and the current GPU resources.

[0100] In one feasible implementation, the allocation of GPU resources corresponding to the target GPU device is paused before the conversion unit 503 converts the driver currently loaded on the target GPU device into the target GPU driver. After the conversion unit 503 converts the driver currently loaded on the target GPU device into the target GPU driver, it starts the function of allocating GPU resources corresponding to each GPU device.

[0101] In one feasible implementation, the target GPU driver allocation information includes: Information on the number of target GPU drivers, or the ratio of target GPU drivers.

[0102] In one feasible implementation, the determining unit 501 is further configured to determine each first GPU device currently loading the first GPU driver, and each second GPU device currently loading the second GPU driver; The conversion unit 503 is also used to convert the GPU driver currently loaded by the idle first GPU device into the second GPU driver when all the second GPU devices are in a non-idle state and there is an idle device among the first GPU devices.

[0103] It should be noted that the information interaction and execution process between the modules / units in each of the above devices are different from those in this application. Figures 1 to 4 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0104] The following describes an electronic device provided by an embodiment of this application. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device, which includes a processor 601 and a memory 600. The memory 600 stores computer-executable instructions that can be executed by the processor 601. The processor 601 executes the computer-executable instructions to implement the methods implemented on the server, or on the labeling terminal, or on the business terminal.

[0105] exist Figure 6 In the illustrated embodiment, the electronic device further includes a bus 602 and a communication interface 603, wherein the processor 601, the communication interface 603, and the memory 600 are connected via the bus 602.

[0106] The memory 600 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 602 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus 102 can be divided into an address bus, a data bus, and a control bus. For ease of representation, Figure 6 The symbol is represented by a single rectangular bar, but this does not mean that there is only one bus or one type of bus.

[0107] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor 601 reads the information in the memory and, in conjunction with its hardware, completes the steps of the method in the aforementioned embodiment.

[0108] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope defined in the claims of the present invention.

Claims

1. A resource allocation method, characterized in that, Applied to an electronic device, the electronic device including multiple GPU devices, each of the GPU devices having at least two different GPU drivers installed, the method includes: Determine the target GPU resource allocation information, which includes target GPU driver allocation information; the target GPU driver allocation information is used to indicate the demand for different GPU drivers, that is, the target GPU driver allocation information needs to be converted into the GPU driver type and the corresponding quantity information. Obtain the driver currently loaded on each GPU device to obtain the current GPU driver allocation information; the GPU driver allocation information is used to indicate the different drivers loaded on different devices, that is, the type of GPU driver and the number of GPUs corresponding to each type of GPU driver; The target GPU driver allocation information is compared with the current GPU driver allocation information to obtain the difference information between the demand for various GPU drivers and the number of devices currently loading various GPU drivers. Based on the difference information, the target GPU driver to be added and the demand for the target GPU driver are determined. Determine at least one first candidate GPU device that is in an idle state from the plurality of GPU devices; Based on the target GPU driver to be added and the demand for the target GPU driver, a target GPU device for driver conversion and a target GPU driver converted from the first candidate GPU devices are determined from each of the first candidate GPU devices. If the number of the first candidate GPU devices is greater than or equal to the demand for the target GPU driver, the GPU device with the demand for the target GPU driver is selected from the first candidate GPU devices as the target GPU device for driver conversion; If the number of the first candidate GPU devices is less than the demand of the target GPU driver, each of the first candidate GPU devices is determined as the target GPU device, and when a second candidate GPU device in an idle state is detected, the second candidate GPU device is determined as the target GPU device, until the number of the target GPU devices is equal to the demand of the target GPU driver; The driver currently loaded on the target GPU device is converted to the target GPU driver, so that the GPU resource allocation is consistent with the resource allocation in the preset resource allocation information.

2. The method according to claim 1, characterized in that, The determination of the target GPU driver allocation information includes: In response to a driver setting instruction, the target GPU driver allocation information is determined, and the driver setting instruction includes the requirements for different GPU drivers. or, The target GPU driver allocation information is determined based on the GPU resource requirements of the application to be run and the current GPU resources.

3. The method according to claim 1, characterized in that, Before converting the driver currently loaded on the target GPU device into the target GPU driver, the method further includes: Pause the allocation of GPU resources corresponding to each of the aforementioned GPU devices; After converting the driver currently loaded on the target GPU device into the target GPU driver, the method further includes: Enable the function of allocating GPU resources corresponding to each of the aforementioned GPU devices.

4. The method according to claim 1, characterized in that, The target GPU driver allocation information includes: Information on the number of target GPU drivers, or the ratio of target GPU drivers.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Identify each first GPU device currently loading the first GPU driver, and each second GPU device currently loading the second GPU driver; When all the second GPU devices are in a non-idle state, and there is an idle device among the first GPU devices, the GPU driver currently loaded on the idle first GPU device is converted to the second GPU driver.

6. A resource allocation device, characterized in that, An application in an electronic device, the electronic device including multiple GPU devices, each of the GPU devices having at least two different GPU drivers installed, the device comprising: A determining unit is used to determine target GPU resource allocation information, which includes target GPU driver allocation information. The target GPU driver allocation information is used to indicate the demand for different GPU drivers, i.e., the target GPU driver allocation information needs to be converted into GPU driver type and corresponding quantity information. The acquisition unit is used to acquire the driver currently loaded by each GPU device and obtain the current GPU driver allocation information; the GPU driver allocation information is used to indicate the different drivers loaded by different devices, that is, the type of GPU driver and the number of GPUs corresponding to each type of GPU driver; The determining unit is further configured to compare the target GPU driver allocation information with the current GPU driver allocation information to obtain the difference information between the demand for various GPU drivers and the number of devices currently loading various GPU drivers, and determine the target GPU driver to be added and the demand for the target GPU driver based on the difference information. The determining unit is further configured to determine at least one first candidate GPU device that is in an idle state from the plurality of GPU devices; and to determine, based on the target GPU driver to be added and the demand for the target GPU driver, a target GPU device for driver conversion and a target GPU driver converted from each of the first candidate GPU devices; The determining unit is further configured to: when the number of the first candidate GPU devices is greater than or equal to the demand of the target GPU driver, select the GPU device with the demand of the target GPU driver from the first candidate GPU devices as the target GPU device for driver conversion; and when the number of the first candidate GPU devices is less than the demand of the target GPU driver, determine each of the first candidate GPU devices as the target GPU device, and when a second candidate GPU device in an idle state is detected, determine the second candidate GPU device as the target GPU device, until the number of the target GPU devices is equal to the demand of the target GPU driver; The conversion unit is used to convert the driver currently loaded on the target GPU device into the target GPU driver, so that the GPU resource allocation is consistent with the resource allocation in the preset resource allocation information.

7. An electronic device, characterized in that, The electronic device includes multiple GPU devices, each of which has at least two different GPU drivers installed. The electronic device includes: A memory and a processor, wherein the memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is used to execute the one or more computer instructions to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing one or more computer instructions thereon, characterized in that, The method is applied to an electronic device that includes multiple GPU devices, each of which has at least two different GPU drivers installed, wherein the computer instructions are executed by a processor to implement the method as described in any one of claims 1-5.

Images