A GPU resource allocation method, apparatus, device, and readable storage medium

Abstract

This application discloses a GPU resource allocation method, apparatus, device, and readable storage medium. The method includes: determining whether the instruction currently being executed by the target program is a CUDA instruction with an attached eBPF tag; if so, triggering an eBPF process to perform an execution audit of GPU resource quantity through the eBPF tag attached to the CUDA instruction; obtaining the execution audit result returned by the eBPF process, the execution audit result being determined based on the rated GPU resource quantity corresponding to the currently executed instruction and the current real-time resource quantity occupied by the GPU; if the execution audit result is successful, allowing GPU resource allocation by calling the currently executed instruction; if the execution audit result is unsuccessful, restricting GPU resource allocation according to a preset isolation mode. This application can simultaneously adapt to various CUDA versions, avoiding situations where high performance overhead, long development cycles, and high maintenance costs affect upper-layer applications.

Description

Technical Field

[0001] This application relates to the field of computer intelligence, and more specifically, to a GPU resource allocation method, apparatus, device, and readable storage medium. Background Technology

[0002] With the continuous development of AI technology, GPUs (Graphics Processing Units) are widely used in deep learning. Existing GPU processors often contain over 10GB of video memory and up to 2000 GPU computing cores, such as the Nvidia V100. Therefore, coarse-grained GPU scheduling, where a single task exclusively uses the GPU, leads to significant waste of GPU resources. To improve GPU resource utilization and task throughput, GPU virtualization has gradually become a hot topic in the industry. GPU virtualization refers to logically dividing a physical GPU into multiple virtual GPUs, using the virtual GPUs as scheduling units, to achieve the goal of multiple services sharing a single physical GPU.

[0003] Regarding GPU sharing and isolation, both academia and major manufacturers have proposed corresponding solutions, which mainly include the following:

[0004] One approach is to hijack the CUDA API to limit GPU resources. However, this method requires modifying and replacing CUDA instructions based on the CUDA version, which can impact upper-layer GPU applications. Furthermore, different CUDA versions require different hijacking schemes, resulting in very high development and maintenance costs.

[0005] Second, a time-division multiplexing approach is used to schedule GPU tasks to run on the GPU in a time-slice round-robin manner, achieving the goal of sharing the GPU. This solution also requires custom development of CUDA, resulting in poor compatibility and maintainability. Furthermore, in this type of time-division multiplexing scheme, the GPU context switching overhead caused by task switching is significant, leading to considerable performance degradation.

[0006] Thirdly, a proposal was made to modify deep learning frameworks to allow multiple deep learning tasks to run simultaneously on a single GPU. However, developing and maintaining solutions for modifying deep learning frameworks is difficult, and modifications can only be provided for a specific version of a specific framework. This carries a high risk of incompatibility with user code and is not suitable for most user GPU tasks. Therefore, this solution has limited practical value.

[0007] Based on the above, the incompatibility of CUDA versions causes significant problems for GPU sharing and isolation. This application provides a GPU resource allocation scheme that can adapt to various CUDA versions simultaneously, avoiding high performance overhead, long development cycles, high maintenance costs, and impact on upper-layer applications. Summary of the Invention

[0008] In view of this, this application provides a GPU resource allocation method, apparatus, device, and readable storage medium that can be adapted to different CUDA versions and has low performance overhead, short development cycle, and low maintenance cost.

[0009] To achieve the above objectives, the following solution is proposed:

[0010] A GPU resource allocation method, comprising:

[0011] Determine whether the instruction currently being executed by the target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction with an eBPF tag mounted;

[0012] If so, the eBPF process is triggered by the eBPF tag attached to the CUDA instruction to perform an execution audit of GPU resource quantity;

[0013] Obtain the execution audit result returned by the eBPF process. The execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the current real-time resource amount occupied by the GPU.

[0014] If the execution audit result is "audit passed", then GPU resource allocation is allowed by calling the currently executed instruction;

[0015] If the audit result is that the audit fails, then the GPU resource allocation will be restricted according to the preset isolation mode.

[0016] Preferably, the process of attaching the eBPF tag to the resource allocation CUDA instruction includes:

[0017] Get the current CUDA version;

[0018] Read the logical address of the resource allocation CUDA instruction corresponding to the current CUDA version from the stored configuration file;

[0019] The eBPF tag is attached to the logical address of the resource allocation CUDA instruction.

[0020] Preferably, the process by which the eBPF process determines the returned execution audit result includes:

[0021] The eBPF process determines the current amount of idle GPU resources based on the current real-time resource usage of the GPU and the preset limit on the amount of GPU resources allowed to be used.

[0022] The eBPF process determines whether the rated GPU resource amount corresponding to the currently executed instruction exceeds the current GPU idle resource amount;

[0023] If so, the execution audit result returned by the eBPF process will be "audit failed"; otherwise, it will be "audit passed".

[0024] Preferably, the allocation restriction according to the preset isolation mode includes:

[0025] GPU resource allocation is limited by GPU hard isolation;

[0026] or,

[0027] GPU resource allocation is restricted by GPU soft isolation.

[0028] Preferably, the step of limiting GPU resource allocation according to GPU hard isolation includes:

[0029] The GPU resource allocation process will be temporarily suspended, and a message indicating insufficient resource allocation will be displayed.

[0030] Preferably, the step of limiting GPU resource allocation according to GPU soft isolation includes:

[0031] Determine if the current amount of free GPU resources is empty;

[0032] If so, then postpone this GPU resource allocation.

[0033] If not, GPU resource allocation is permitted by invoking the currently executing instruction.

[0034] Preferably, the resource allocation CUDA instructions include video memory allocation CUDA instructions, computing power allocation CUDA instructions, and device information CUDA instructions.

[0035] A GPU resource allocation device, comprising:

[0036] The instruction determination unit is used to determine whether the instruction currently being executed by the target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction with an eBPF tag attached.

[0037] The execution audit unit is used to trigger the eBPF process to perform execution audit of GPU resource quantity when the instruction currently being executed by the target program is a CUDA instruction. This is done by using the eBPF tag attached to the CUDA instruction.

[0038] The result acquisition unit is used to acquire the execution audit result returned by the eBPF process. The execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the resource amount currently occupied by the GPU in real time.

[0039] A resource allocation unit is configured to allow GPU resource allocation by invoking the currently executing instruction if the execution audit result is successful.

[0040] The resource restriction unit is used to restrict GPU resource allocation according to a preset isolation mode when the audit result is that the audit fails.

[0041] A GPU resource allocation device, including memory and a processor;

[0042] The memory is used to store programs;

[0043] The processor is used to execute the program and implement the various steps of the GPU resource allocation method described above.

[0044] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the GPU resource allocation method described above.

[0045] As can be seen from the above technical solution, the GPU resource allocation method, apparatus, device, and readable storage medium provided in this application determine whether the instruction currently being executed by the target program is a CUDA instruction, wherein the CUDA instruction is a resource allocation CUDA instruction with an eBPF tag attached. If the currently executed instruction is a CUDA instruction, the eBPF process is triggered to perform an execution audit of the GPU resource quantity by means of the eBPF tag attached to the CUDA instruction. The execution audit result returned by the eBPF process is obtained, and the execution audit result is determined based on the rated GPU resource quantity corresponding to the currently executed instruction and the current real-time resource quantity occupied by the GPU. If the execution audit result is that the audit passes, GPU resource allocation is allowed by calling the currently executed instruction; if the execution audit result is that the audit fails, GPU resource allocation is restricted according to a preset isolation mode.

[0046] This application employs an instruction-triggered mechanism. The eBPF process is triggered via an eBPF tag only when a resource allocation CUDA instruction is executed, thereby enabling execution auditing of GPU resource usage and subsequent GPU resource allocation limits. Since the eBPF process runs within the kernel, it eliminates significant overhead such as data copying between user space and kernel space, system calls, and context switching, resulting in low performance overhead. This application achieves GPU resource allocation limits by attaching the eBPF process to the resource allocation CUDA instruction, without requiring modification to any CUDA instructions or other system code. Furthermore, various CUDA versions have corresponding resource allocation CUDA instructions; therefore, this application is adaptable to various CUDA versions, eliminating the need for separate development for each CUDA version. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of an eBPF tag mounting process according to an embodiment of this application;

[0049] Figure 2 This is a flowchart of a GPU resource allocation method disclosed in this application;

[0050] Figure 3 This is a schematic diagram illustrating a GPU resource allocation example from this application;

[0051] Figure 4 This is a schematic diagram illustrating a GPU task processing example from this application;

[0052] Figure 5 This is a block diagram of a GPU resource allocation device disclosed in this application;

[0053] Figure 6 This is a hardware structure block diagram of a GPU resource allocation device disclosed in this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.

[0056] Before explaining the specific implementation process of GPU resource allocation in this application, we will first introduce the process of attaching the eBPF tag to the resource allocation CUDA instruction.

[0057] This application implements GPU resource allocation based on the concept of CUDA hijacking, introducing eBPF technology to hijack CUDA. Since the CUDA Driver library is also a user-space program, eBPF tags can be used to bind resource allocation-related instructions within it. By hijacking GPU resource allocation through the eBPF process, the goal of controlling GPU resource allocation can be achieved.

[0058] This application utilizes eBPF, a top-level submodule in the Linux kernel. It can be simply understood as a small virtual machine within the kernel, allowing kernel code written in C to be injected into and executed from user space. eBPF enables flexible modification of kernel processing strategies without altering the kernel code. The eBPF process is run by the kernel when an event is triggered; it's a form of function hooking or event-driven programming. eBPF tags can be mounted to specified mount points in the CUDA Driver dynamic library libcuda.so. Mount points can be system calls or instructions in user-space programs, and each mount point is bound to an eBPF tag line. An eBPF process can have multiple eBPF tags and can be bound to multiple mount points. When the target program executes a mounted instruction, it jumps to execute the eBPF process, and after execution, jumps back to the mount point to continue running the target program. The entire execution process requires no intrusive modification to the target program.

[0059] The following is combined Figure 1 This section describes the process of attaching eBPF tags to resource allocation CUDA instructions, which may include:

[0060] First, obtain the current CUDA version.

[0061] Specifically, while the resource allocation instructions are the same across different CUDA versions, their logical addresses differ. The loader needs to automatically scan the local CUDA version to determine the logical address of the resource allocation instructions for the current CUDA version.

[0062] Secondly, the logical address of the resource allocation CUDA instruction corresponding to the current CUDA version is read from the stored configuration file.

[0063] Specifically, the configuration file stores the logical addresses of resource allocation-related instructions in each CUDA version, i.e., the logical addresses of the resource allocation CUDA instructions that need to be hijacked. The loader scans to obtain the local CUDA version and reads the logical address information of the resource allocation CUDA instructions corresponding to the current CUDA version from the configuration file.

[0064] Then, the eBPF tag is attached to the logical address of the resource allocation CUDA instruction.

[0065] Specifically, the eBPF tag is attached to the logical address of the CUDA instruction that allocates resources for the current CUDA version. After attachment, when a user program calls the corresponding instruction in the CUDA library to allocate resources, the eBPF process is triggered through the attached eBPF tag, thereby controlling the resource allocation.

[0066] It is understandable that after attaching the eBPF tag to the resource allocation CUDA instruction, when the user program executes the resource allocation CUDA instruction in the CUDA library for resource allocation, the eBPF process will be triggered through the attached eBPF tag, thereby controlling the resource allocation process and achieving the purpose of resource limitation. The logical address of the resource allocation CUDA instruction corresponding to the current CUDA version is read from the stored configuration file to implement the eBPF tag attachment, enabling this application to adapt to various CUDA version environments. It is only necessary to obtain the logical address of the resource allocation CUDA instruction for each version of the CUDA library, summarize and form a configuration file, store the configuration file, and identify and read the configuration file during attachment to achieve GPU resource allocation in various CUDA version environments.

[0067] Figure 2 This is a flowchart of a GPU resource allocation method disclosed in an embodiment of this application. Based on the above process of attaching eBPF tags to resource allocation CUDA instructions, the GPU resource allocation method is described. See [link to relevant documentation]. Figure 2 The method may include:

[0068] Step S1: Determine whether the instruction currently being executed by the target program is a CUDA instruction.

[0069] Specifically, the CUDA instruction to be mounted is the resource allocation CUDA instruction with the eBPF tag attached. Since the process of attaching the eBPF tag to the resource allocation CUDA instruction has already been completed before GPU resource allocation, it is only necessary to determine whether the instruction currently being executed by the target program is a mounted CUDA instruction to determine whether the instruction currently being executed by the target program is the resource allocation related instruction that we need to restrict.

[0070] If the currently executed instruction is a CUDA mounting instruction, proceed to step S2: trigger the eBPF process to perform an execution audit of GPU resource quantities by using the eBPF tag attached to the CUDA mounting instruction.

[0071] Specifically, such as Figure 3 As shown, the target program sequentially calls and executes programs in libcuda.so. When the currently executing instruction is a CUDA mounting instruction, it triggers the eBPF process through the mounted eBPF tag to perform execution auditing on GPU resources. The eBPF process collects CUDA execution parameters and compares the rated GPU resource amount corresponding to the currently executing instruction (i.e., the user task) with the currently real-time GPU resource amount to determine whether to approve the current resource allocation request, i.e., to determine the execution audit.

[0072] The eBPF process can specifically include the eBPF program, eBPF Maps, and a control program. The eBPF program can collect CUDA execution parameters, such as the currently allocated GPU memory and the number of SMs requested, and send them to the control program through the eBPF Maps. The control program compares the rated GPU resource amount with the GPU's real-time resource usage to determine whether to approve the current resource allocation request. The audit result is sent to the eBPF program through the eBPF Maps, and the eBPF program executes the audit result. If it passes, it normally calls CUDA instructions to allocate resources; otherwise, it restricts the current resource allocation.

[0073] Step S3: Obtain the execution audit results returned by the eBPF process.

[0074] Specifically, the execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the current real-time resource usage of the GPU. The execution audit result includes audit pass and audit fail. Figure 3 As shown, the eBPF process returns and executes the audit results. If the audit passes, the currently executing CUDA instruction is allowed to be called normally for resource allocation; otherwise, resource allocation is restricted.

[0075] If the audit result is that the audit is passed, proceed to step S4, which allows GPU resource allocation by calling the currently executed instruction.

[0076] If the audit result is that the audit fails, proceed to step S5 and restrict GPU resource allocation according to the preset isolation mode.

[0077] Specifically, there are two types of isolation modes: hard isolation and soft isolation. Hard isolation means that the amount of resources currently used by the GPU has reached its limit, and the tasks related to this resource allocation will be interrupted and a message indicating insufficient resources will be displayed. Soft isolation means that the amount of resources currently used by the GPU has reached its limit, and it is necessary to further determine whether there are any idle resources available for the GPU and appropriately relax the resource restrictions on the GPU.

[0078] As can be seen from the above technical solution, the GPU resource allocation method provided in this application determines whether the instruction currently being executed by the target program is a CUDA instruction, wherein the CUDA instruction is a resource allocation CUDA instruction with an eBPF tag attached. If the currently executed instruction is a CUDA instruction, the eBPF process is triggered to perform an execution audit of the GPU resource quantity by means of the eBPF tag attached to the CUDA instruction. The execution audit result returned by the eBPF process is obtained, and the execution audit result is determined based on the rated GPU resource quantity corresponding to the currently executed instruction and the current real-time resource quantity occupied by the GPU. If the execution audit result is that the audit passes, GPU resource allocation is allowed by calling the currently executed instruction; if the execution audit result is that the audit fails, GPU resource allocation is restricted according to a preset isolation mode.

[0079] This application employs an instruction-triggered mechanism. The eBPF process is triggered via an eBPF tag only when a resource allocation CUDA instruction is executed, thereby enabling execution auditing of GPU resource usage and subsequent GPU resource allocation limits. Since the eBPF process runs within the kernel, it eliminates significant overhead such as data copying between user space and kernel space, system calls, and context switching, resulting in low performance overhead. This application achieves GPU resource allocation limits by attaching the eBPF process to the resource allocation CUDA instruction, without requiring modification to any CUDA instructions or other system code. Furthermore, various CUDA versions have corresponding resource allocation CUDA instructions; therefore, this application is adaptable to various CUDA versions, eliminating the need for separate development for each CUDA version.

[0080] Optionally, resource allocation CUDA instructions may include memory allocation CUDA instructions, computing power allocation CUDA instructions, and device information CUDA instructions.

[0081] Specifically, referring to Table 1, resource allocation CUDA instructions can be divided into memory allocation CUDA instructions, computing power allocation CUDA instructions, and device information CUDA instructions. The CUDA Driver API column in Table 1 lists several memory allocation CUDA instructions, computing power allocation CUDA instructions, and device information CUDA instructions respectively.

[0082] When executing a CUDA instruction for memory allocation, the system summarizes and compares the currently used GPU memory with the allocated resource amount specified in the CUDA instruction during task creation to determine whether to allow the resource allocation request. If the audit passes, the memory allocation is initiated normally; otherwise, allocation is restricted according to the specified isolation mode.

[0083] When executing the CUDA instruction for computing power allocation, the total number of SMs currently used by the GPU is summarized. If the usage is below the limit and the audit passes, the kernel function is called normally to start computing; otherwise, the allocation is limited according to the specified isolation mode.

[0084] When the API related to device information is executed, the current total GPU memory will be replaced with the amount of GPU resources specified when the task was created, and the current free memory will be replaced with the difference between the allocated memory and the used memory for the task.

[0085]

[0086] Table 1

[0087] In some embodiments of this application, the process of the eBPF process determining the returned execution audit result in step S3 is described, which may specifically include:

[0088] Step S31: The eBPF process determines the current amount of idle GPU resources based on the current real-time resource usage of the GPU and the preset upper limit of allowed GPU resource usage.

[0089] Specifically, the eBPF process first obtains the amount of resources currently occupied by the GPU in real time and the preset limit of allowed GPU resources. The difference between the preset limit of allowed GPU resources and the amount of resources currently occupied by the GPU in real time is the amount of free GPU resources.

[0090] Step S32: The eBPF process determines whether the rated GPU resource amount corresponding to the currently executed instruction exceeds the current GPU idle resource amount;

[0091] If so, the execution audit result returned by the eBPF process will be "audit failed"; otherwise, it will be "audit passed".

[0092] Specifically, the system retrieves the rated GPU resource amount corresponding to the currently executed instruction and determines whether the rated GPU resource amount exceeds the current GPU idle resource amount. If so, the audit result is "audit fails"; otherwise, the audit passes.

[0093] In some embodiments of this application, two optional implementation methods are provided for step S5, the process of restricting GPU resource allocation according to a preset isolation mode. These two optional implementation methods are described below and may specifically include:

[0094] The first method is to restrict GPU resource allocation based on GPU hard isolation.

[0095] Specifically, in the case of hard isolation, if the execution audit result returned by the eBPF process is that the audit fails, the process of limiting GPU resource allocation in GPU hard isolation includes suspending the execution of this GPU resource allocation and prompting that the resource allocation is insufficient.

[0096] The second method is to limit GPU resource allocation based on GPU soft isolation.

[0097] The process of GPU soft isolation limiting GPU resource allocation includes:

[0098] Determine if the current amount of free GPU resources is empty;

[0099] If so, then postpone this GPU resource allocation.

[0100] If not, GPU resource allocation is permitted by invoking the currently executing instruction.

[0101] Specifically, in the case of soft isolation, if the execution audit result returned by the eBPF process is "audit failed," it is further determined whether the current GPU idle resource amount is empty. If the GPU idle resource amount is empty, the execution of this GPU resource allocation is temporarily suspended. If the GPU idle resource amount is not empty, it means that the current GPU idle resource amount does not meet the rated GPU resource amount corresponding to the currently executed instruction, but the current real-time resource usage of the GPU has not reached the preset allowed GPU resource limit. In this case, the GPU resource limit can be appropriately relaxed in the soft isolation scenario, allowing GPU resource allocation by calling the currently executed instruction.

[0102] It is understood that the implementation methods for isolating and restricting GPU resource allocation in this application include, but are not limited to, the two methods mentioned above. This application emphasizes that when the execution audit result is that the audit fails, that is, the rated GPU resource amount corresponding to the currently executed instruction exceeds the current GPU idle resource amount, the GPU resource allocation is isolated and restricted. Any preset isolation restriction method that conforms to reality should be within the protection scope of this application.

[0103] The following is combined Figure 4 This paper illustrates the practical application of this application through a GPU task processing example.

[0104] Initially, during system operation, the shared module reads the local GPU information and divides a single physical GPU into 100 resource units. Each GPU resource unit then possesses 1 / 100 of the video memory and computing power resources. Video memory is divided by 100 (total video memory), and computing power is divided by 100 (number of SMs). For example, if a physical GPU has 16000MB of video memory and 80 SMs, then after division, one GPU resource unit represents 1600MB of video memory and 8 SMs. Since executing CUDA instructions for GPU computation essentially involves specifying an SM to run a kernel function, and each SM contains a large number of basic arithmetic instruction execution units (SPs), the actual computation is performed by the SPs. Therefore, this application proposes that the quantification of GPU computing power can be defined by the number of SMs, and the computing power allocation ratio can be proportionally controlled by controlling the number of SMs allocated to tasks.

[0105] After a user creates a GPU task, the corresponding CUDA instructions are generated and placed in libcuda.so for execution. The shared module, based on the GPU sharing scheduling logic, specifies the GPU ID, rated GPU resources, isolation mode, etc., when creating the GPU task. The GPU ID refers to the GPU ID in the host machine, numbered starting from 0; for example, GPU number 0 is the first GPU card. The rated GPU resources refer to the estimated amount of GPU resources required for the task; for example, 20 GPU resources means 20 / 100 GPUs. Isolation modes include hard isolation and soft isolation.

[0106] The sharing module determines whether to allocate GPUs to tasks in progress based on the current GPU resource availability. For example, if one GPU has 100 GPU resource shares and each task requests 20 GPU shares, then that physical GPU can be allocated to a maximum of 5 tasks. No other task will request GPU allocation until these 5 tasks have completed. After GPU allocation is complete, the sharing module stores information such as the task ID, the bound GPU ID, the rated GPU resource amount, and the isolation mode in the database for archiving.

[0107] When instructions in libcuda.so are executed one by one, the eBPF process is triggered when a resource allocation CUDA instruction with an attached eBPF process is encountered. The control program retrieves the rated GPU resource amount for the current resource allocation CUDA instruction from the database, determines whether to execute the instruction, and sends the audit result to the eBPF program via eBPF Maps. If the audit result is successful, the eBPF program allows the execution of the resource allocation CUDA instruction and performs subsequent GPU resource allocation according to the predetermined GPU ID and rated GPU resource amount. If the audit result is unsuccessful, the eBPF program restricts subsequent GPU resource allocation through a predetermined isolation mode.

[0108] The GPU resource allocation apparatus provided in the embodiments of this application is described below. The GPU resource allocation apparatus described below can be referred to in correspondence with the GPU resource allocation method described above.

[0109] See Figure 5 , Figure 5 This is a schematic diagram of a GPU resource allocation device disclosed in an embodiment of this application.

[0110] like Figure 5 As shown, the device may include:

[0111] The instruction determination unit is used to determine whether the instruction currently being executed by the target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction with an eBPF tag attached.

[0112] The execution audit unit is used to trigger the eBPF process to perform execution audit of GPU resource quantity when the instruction currently being executed by the target program is a CUDA instruction. This is done by using the eBPF tag attached to the CUDA instruction.

[0113] The result acquisition unit is used to acquire the execution audit result returned by the eBPF process. The execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the resource amount currently occupied by the GPU in real time.

[0114] A resource allocation unit is configured to allow GPU resource allocation by invoking the currently executing instruction if the execution audit result is successful.

[0115] The resource restriction unit is used to restrict GPU resource allocation according to a preset isolation mode when the audit result is that the audit fails.

[0116] As can be seen from the above technical solution, the GPU resource allocation device provided in this application determines whether the instruction currently being executed by the target program is a CUDA instruction, wherein the CUDA instruction is a resource allocation CUDA instruction with an eBPF tag attached. If the currently executed instruction is a CUDA instruction, the eBPF process is triggered to perform an execution audit of the GPU resource quantity by means of the eBPF tag attached to the CUDA instruction. The execution audit result returned by the eBPF process is obtained, and the execution audit result is determined based on the rated GPU resource quantity corresponding to the currently executed instruction and the current real-time resource quantity occupied by the GPU. If the execution audit result is that the audit passes, GPU resource allocation is allowed by calling the currently executed instruction; if the execution audit result is that the audit fails, GPU resource allocation is restricted according to a preset isolation mode.

[0117] This application employs an instruction-triggered mechanism. The eBPF process is triggered via an eBPF tag only when a resource allocation CUDA instruction is executed, thereby enabling execution auditing of GPU resource usage and subsequent GPU resource allocation limits. Since the eBPF process runs within the kernel, it eliminates significant overhead such as data copying between user space and kernel space, system calls, and context switching, resulting in low performance overhead. This application achieves GPU resource allocation limits by attaching the eBPF process to the resource allocation CUDA instruction, without requiring modification to any CUDA instructions or other system code. Furthermore, various CUDA versions have corresponding resource allocation CUDA instructions; therefore, this application is adaptable to various CUDA versions, eliminating the need for separate development for each CUDA version.

[0118] Optionally, the aforementioned GPU resource allocation device may further include a tag mounting unit for performing:

[0119] Get the current CUDA version;

[0120] Read the logical address of the resource allocation CUDA instruction corresponding to the current CUDA version from the stored configuration file;

[0121] The eBPF tag is attached to the logical address of the resource allocation CUDA instruction.

[0122] Optionally, the resource limiting unit mentioned above may include a hard isolation unit or a soft isolation unit;

[0123] Hard isolation units are used to restrict GPU resource allocation according to GPU hard isolation;

[0124] Soft isolation units are used to limit GPU resource allocation according to GPU soft isolation.

[0125] Optionally, the aforementioned hard isolation unit can be used to postpone the execution of this GPU resource allocation and indicate insufficient resource allocation.

[0126] Optionally, the soft isolation unit described above can be used to perform:

[0127] Determine if the current amount of free GPU resources is empty;

[0128] If so, then postpone this GPU resource allocation.

[0129] If not, GPU resource allocation is permitted by invoking the currently executing instruction.

[0130] Optionally, the above-mentioned resource allocation CUDA instructions may include video memory allocation CUDA instructions, computing power allocation CUDA instructions, and device information CUDA instructions.

[0131] The GPU resource allocation device provided in this application embodiment can be applied to GPU resource allocation devices. Optionally, Figure 6 The hardware structure block diagram of the GPU resource allocation device is shown below. Figure 6 The hardware structure of a GPU resource allocation device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0132] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0133] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0134] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0135] The memory stores a program, which the processor can call. The program is used for:

[0136] Determine whether the instruction currently being executed by the target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction with an eBPF tag mounted;

[0137] If so, the eBPF process is triggered by the eBPF tag attached to the CUDA instruction to perform an execution audit of GPU resource quantity;

[0138] Obtain the execution audit result returned by the eBPF process. The execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the current real-time resource amount occupied by the GPU.

[0139] If the execution audit result is "audit passed", then GPU resource allocation is allowed by calling the currently executed instruction;

[0140] If the audit result is that the audit fails, then the GPU resource allocation will be restricted according to the preset isolation mode.

[0141] Optionally, the refined and extended functions of the program can be referred to the above description.

[0142] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0143] Determine whether the instruction currently being executed by the target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction with an eBPF tag mounted;

[0144] If so, the eBPF process is triggered by the eBPF tag attached to the CUDA instruction to perform an execution audit of GPU resource quantity;

[0145] Obtain the execution audit result returned by the eBPF process. The execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the current real-time resource amount occupied by the GPU.

[0146] If the execution audit result is "audit passed", then GPU resource allocation is allowed by calling the currently executed instruction;

[0147] If the audit result is that the audit fails, then the GPU resource allocation will be restricted according to the preset isolation mode.

[0148] Optionally, the refined and extended functions of the program can be referred to the above description.

[0149] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0151] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for GPU resource allocation, the method comprising: The method comprises the following steps: determining whether the currently executed instruction of a target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction mounted with an eBPF label; if yes, triggering an eBPF process to perform an execution audit on the GPU resource amount through the eBPF label mounted on the mounted CUDA instruction; obtaining an execution audit result returned by the eBPF process, wherein the execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the real-time occupied resource amount of the GPU; if the execution audit result is an audit pass, allowing GPU resource allocation through the currently executed instruction; if the execution audit result is an audit fail, limiting GPU resource allocation according to a preset isolation mode; wherein the process of mounting the eBPF label to the resource allocation CUDA instruction comprises the following steps: obtaining a current CUDA version; reading the logical address of the resource allocation CUDA instruction corresponding to the current CUDA version from a stored configuration file; mounting the eBPF label to the logical address of the resource allocation CUDA instruction.

2. The method of claim 1, wherein, The process of determining the returned execution audit result by the eBPF process comprises the following steps: the eBPF process determines the current GPU idle resource amount according to the real-time occupied resource amount of the GPU and a preset upper limit of the allowed use of GPU resources; the eBPF process determines whether the rated GPU resource amount corresponding to the currently executed instruction exceeds the current GPU idle resource amount; if yes, the execution audit result returned by the eBPF process is an audit fail, otherwise, it is an audit pass.

3. The method of claim 1, wherein, The limitation of allocation according to the preset isolation mode comprises the following steps: limiting GPU resource allocation according to GPU hard isolation; or, limiting GPU resource allocation according to GPU soft isolation.

4. The method of claim 3, wherein, The limitation of allocation according to GPU hard isolation comprises the following steps: suspending the current GPU resource allocation and prompting resource allocation shortage.

5. The method of claim 3, wherein, The limitation of allocation according to GPU soft isolation comprises the following steps: determining whether the current GPU idle resource amount is empty; if yes, suspending the current GPU resource allocation; if no, allowing GPU resource allocation through the currently executed instruction.

6. The method of claim 1, wherein, The resource allocation CUDA instruction comprises a video memory allocation CUDA instruction, a computing power allocation CUDA instruction, and a device information CUDA instruction.

7. A GPU resource allocation apparatus, characterized by, The method comprises the following steps: an instruction determination unit is configured to determine whether the currently executed instruction of a target program is a mounted CUDA instruction, wherein the mounted CUDA instruction is a resource allocation CUDA instruction mounted with an eBPF label; an execution audit unit is configured to trigger an eBPF process to perform an execution audit on the GPU resource amount through the eBPF label mounted on the mounted CUDA instruction when the currently executed instruction of the target program is the mounted CUDA instruction; a result obtaining unit is configured to obtain an execution audit result returned by the eBPF process, wherein the execution audit result is determined based on the rated GPU resource amount corresponding to the currently executed instruction and the real-time occupied resource amount of the GPU. A resource allocation unit is configured to allow GPU resource allocation by invoking the instruction of the current execution when the execution audit result is audit pass. A resource limiting unit is configured to limit GPU resource allocation according to a preset isolation mode when the execution audit result is audit fail. The process of mounting the eBPF label to the resource allocation CUDA instruction includes: Obtaining a current CUDA version; Reading the logical address of the resource allocation CUDA instruction corresponding to the current CUDA version from a stored configuration file; Mounting the eBPF label to the logical address of the resource allocation CUDA instruction.

8. A GPU resource allocation device, comprising: A memory and a processor are included; The memory is configured to store a program; The processor is configured to execute the program to implement each step of the GPU resource allocation method according to any one of claims 1-6.

9. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements each step of the GPU resource allocation method according to any one of claims 1-6.

Images