A resource allocation method, apparatus and electronic device

Abstract

This invention provides a resource allocation method, apparatus, and electronic device, relating to the field of network transmission technology. The method includes: upon receiving a first GPU resource request for a first task, determining whether the first task has resource authorization information for running on the first GPU; if it is determined that the first task does not have resource authorization information for running on the first GPU, determining the resource authorization information for the first task based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the first GPU resource utilization rate, and the number of tasks currently running on the first GPU and the priority level of each task; the waiting queue includes at least one task to be allocated first GPU resources; the resource authorization information includes the time quota for the first task running on the first GPU, which can guarantee the performance of the first priority task, improve GPU resource utilization, and reduce system overhead.

Description

Technical Field

[0001] This invention relates to the field of network transmission technology, and in particular to a resource allocation method, apparatus, and electronic device. Background Technology

[0002] Existing GPU (Graphics Processing Unit) resource allocation methods employ a token scheduling strategy that prioritizes allocating tokens to the task container with the lowest current GPU utilization in the backend waiting queue. Specifically, when multiple tasks simultaneously require GPU access, the backend module allocates tokens to the task with the lowest GPU utilization, delaying computations performed by other tasks on the GPU at that moment. As a result, this existing GPU resource allocation method leads to varying degrees of delays in completion times for multiple tasks sharing the same GPU due to inter-task interference. In other words, the performance of all tasks on the shared GPU cannot be guaranteed, making it unsuitable for use in public clouds. Summary of the Invention

[0003] The purpose of this invention is to provide a resource allocation method, apparatus, and electronic device to solve the problem of extended completion time when running multiple tasks on the same GPU in the public cloud in the prior art.

[0004] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0005] A resource allocation method, comprising:

[0006] When a first graphics processor (GPU) resource request is received for the first task, it is determined whether the first task has resource authorization information running on the first GPU.

[0007] When it is determined that the first task does not have resource authorization information for running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks.

[0008] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0009] Optionally, in the resource allocation method, when it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks, including:

[0010] If the number of tasks in the waiting queue is at least one, determine whether the priority level of the first task is the first priority.

[0011] If the priority level of the first task is the first priority, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0012] Optionally, in the resource allocation method, after determining whether the priority level of the first task is the first priority, the method further includes:

[0013] If the number of tasks in the waiting queue is greater than one, and if the priority level of the first task is not the first priority, and there is no task with the first priority level in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks; the second task is at least one task in the waiting queue other than the first task.

[0014] Optionally, in the resource allocation method, after determining whether the priority level of the first task is the first priority, the method further includes:

[0015] If the number of tasks in the waiting queue is one, and if the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than the first GPU limit, the resource authorization information is determined according to the number of tasks currently running on the first GPU and the priority level of the tasks.

[0016] Optionally, in the resource allocation method, determining the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks includes:

[0017] The time quota is determined according to the following formula:

[0018] T = (1 - k1n1 - k2(n2-1))T max ;

[0019] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0020] Optionally, the resource allocation method further includes:

[0021] When it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource call request is executed, so that the first GPU runs the first task.

[0022] Optionally, in the resource allocation method, when it is determined that the first task has resource authorization information for running on the first GPU, or after determining the resource authorization information of the first task, the first GPU resource call request is executed, and after the first GPU runs the first task, the method further includes:

[0023] Obtain the duration information of the first task running on the first GPU;

[0024] If the duration information exceeds the time quota, the execution of the first GPU resource call request is stopped.

[0025] This invention also provides an electronic device, including a processor and a transceiver, wherein:

[0026] The processor is used to determine whether the first task has resource authorization information running on the first GPU when it receives a first graphics processor GPU resource call request for the first task.

[0027] The processor is further configured to, when determining that the first task does not have resource authorization information running on the first GPU, determine the resource authorization information of the first task based on the number of tasks in the waiting queue, the priority level and graphics processor resource utilization of each task in the waiting queue, and the number of tasks currently running on the first GPU and the priority level of each task.

[0028] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0029] Optionally, in the electronic device, the processor is specifically used for:

[0030] If the number of tasks in the waiting queue is at least one, determine whether the priority level of the first task is the first priority.

[0031] If the priority level of the first task is the first priority, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0032] Optionally, in the electronic device, the processor is specifically used for:

[0033] If the number of tasks in the waiting queue is greater than one, and if the priority level of the first task is not the first priority, and there is no task with the first priority level in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks; the second task is at least one task in the waiting queue other than the first task.

[0034] Optionally, in the electronic device, the processor is specifically used for:

[0035] If the number of tasks in the waiting queue is one, and if the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than the first GPU limit, the resource authorization information is determined according to the number of tasks currently running on the first GPU and the priority level of the tasks.

[0036] Optionally, in the electronic device, the processor is specifically used for:

[0037] The time quota is determined according to the following formula:

[0038] T = (1 - k1n1 - k2(n2-1))T max ;

[0039] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0040] Optionally, in the electronic device, the processor is specifically used for:

[0041] When it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource call request is executed, so that the first GPU runs the first task.

[0042] Optionally, in the electronic device, the processor is specifically used for:

[0043] Obtain the duration information of the first task running on the first GPU;

[0044] If the duration information exceeds the time quota, the execution of the first GPU resource call request is stopped.

[0045] This invention also provides a resource scheduling device, comprising:

[0046] The first judgment module is used to determine whether the first task has resource authorization information running on the first GPU when it receives the first graphics processor GPU resource call request of the first task;

[0047] The first determining module is used to determine the resource authorization information of the first task based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks when it is determined that the first task does not have resource authorization information running on the first GPU.

[0048] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0049] Optionally, in the resource allocation device, the first determining module includes:

[0050] The first judgment unit is used to determine whether the priority level of the first task is the first priority when the number of tasks in the waiting queue is at least one.

[0051] The first determining unit is configured to determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks if the priority level of the first task is the first priority.

[0052] Optionally, the resource allocation device further includes:

[0053] The second determining module is configured to, when the number of tasks in the waiting queue is greater than one, if the priority level of the first task is not the first priority, if there is no task with the priority level of the first priority in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0054] The second task is at least one task in the waiting queue other than the first task.

[0055] Optionally, the resource allocation device further includes:

[0056] The third determining module is used to determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks when the number of tasks in the waiting queue is one, if the priority level of the first task is not the first priority, and if the first GPU resource utilization rate of the first task is less than the first GPU limit.

[0057] Optionally, in the resource allocation device, the first determining module includes:

[0058] The second determining unit is used to determine the time quota according to the following formula:

[0059] T = (1 - k1n1 - k2(n2-1))T max ;

[0060] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0061] Optionally, the resource allocation device further includes:

[0062] The execution module is configured to execute the first GPU resource call request when it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, so that the first GPU runs the first task.

[0063] Optionally, the resource allocation device further includes:

[0064] The acquisition module is used to acquire information about the duration of the first task running on the first GPU;

[0065] The stop module is used to stop executing the first GPU resource call request if the duration information is greater than the time quota.

[0066] This invention also provides an electronic device, including: a transceiver, a processor, a memory, and a program or instructions stored in the memory and executable on the processor; when the processor executes the program or instructions, it implements the resource allocation method as described in any of the preceding embodiments.

[0067] This invention also provides a readable storage medium storing a program or instructions thereon, which, when executed by a processor, implement the resource allocation method as described in any of the preceding claims.

[0068] The beneficial effects of the above-mentioned technical solution of the present invention are as follows:

[0069] The present invention, upon receiving a first GPU resource request for a first task, determines whether the first task has resource authorization information for running on the first GPU. If it is determined that the first task does not have resource authorization information for running on the first GPU, the resource authorization information for the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of each task. This approach ensures the performance of the first priority task, improves GPU resource utilization, and reduces system overhead. Attached Figure Description

[0070] Figure 1 A flowchart illustrating the resource allocation method provided in an embodiment of the present invention;

[0071] Figure 2 This is a schematic diagram illustrating the specific process of the resource allocation method provided in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of the structure of the resource allocation device provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of the structure of an electronic device provided in another embodiment of the present invention. Detailed Implementation

[0075] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0076] To address the issue of prolonged completion time when running multiple tasks on the same GPU in a public cloud, this invention provides a resource allocation method. When a first GPU resource request for a first task is received, the method determines whether the first task has resource authorization information for running on the first GPU. If it is determined that the first task does not have resource authorization information for running on the first GPU, the method determines the resource authorization information for the first task based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and their corresponding priority levels. This method ensures the performance of the first priority task, improves GPU resource utilization, and reduces system overhead.

[0077] It should be noted that the resource allocation method provided in this embodiment of the invention can be applied to electronic devices, the electronic devices including at least one container and a token scheduling module communicatively connected to the at least one container; wherein, the container is a kernel virtualization technology that packages tasks, including the program's runtime code, runtime, system tools, system libraries and settings, and can isolate tasks and resources.

[0078] like Figure 1 As shown, this invention provides a resource allocation method in an embodiment, including:

[0079] Step S101: When a first graphics processor (GPU) resource call request is received for the first task, determine whether the first task has resource authorization information running on the first GPU.

[0080] It should be noted that the first task can be run within the first container.

[0081] Furthermore, the first GPU resource call request may be an API (Application Programming Interface) function related to video memory and computing units in the CUDA (Compute Unified Device Architecture) library of the first container, used to call the first GPU resource; the resource authorization information is a token authorizing the first task to call the first GPU resource for video memory and computing.

[0082] In this embodiment of the invention, the first container forcibly loads the interception library before loading the standard CUDA dynamic link library through the LD_LIBRARY_PATH mechanism of Linux (operating system), so that the first GPU resource call request is not executed. Then the interception library determines whether the first task has resource authorization information running on the first GPU, that is, checks whether the first container holds a token and whether the token has expired.

[0083] Step S102: When it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined according to the number of tasks in the waiting queue, the priority level of each task in the waiting queue and the resource utilization rate of the first GPU, as well as the number of tasks currently running on the first GPU and the priority level of the tasks.

[0084] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0085] It should be noted that the time quota refers to the duration during which the first task can use the first GPU resources for video memory and computation.

[0086] In this embodiment of the invention, when it is determined that the first task does not have resource authorization information running on the first GPU, that is, the first container does not have a token or the token has expired, the first container communicates with the token scheduling module through a TCP (Transmission Control Protocol) socket and sends a token request to the token scheduling module. The token request includes the priority level of the first task, the identity of the first GPU, and the number of tasks running on the first GPU. The token scheduling module then determines the resource authorization information of the first task based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks.

[0087] Furthermore, if the first container has a token and the token has not expired, then the first GPU resource call request is executed.

[0088] In this embodiment of the invention, when a first GPU resource call request for a first task is obtained, it is determined whether the first task has resource authorization information running on the first GPU. If it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks. This can ensure the performance of the first priority task, improve GPU resource utilization, and reduce system overhead.

[0089] Optionally, in the resource allocation method, when it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks, including:

[0090] If the number of tasks in the waiting queue is at least one, determine whether the priority level of the first task is the first priority.

[0091] If the priority level of the first task is the first priority, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0092] In this embodiment of the invention, when the number of tasks in the waiting queue is at least one and the priority level of the first task is the first priority, the token scheduling module can determine the resource authorization information, i.e., the token, according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks, and send the token to the first task.

[0093] Then, after determining the resource authorization information, the token scheduling module will allocate resources to other tasks in the waiting queue besides the first task.

[0094] It should be noted that the priority levels of tasks are divided into first priority and second priority. The first priority is higher than the second priority. The first priority is the highest priority level, which can be understood as high priority, while the second priority can be understood as low priority.

[0095] Correspondingly, tasks are divided into high-priority tasks and low-priority tasks. A GPU can only allocate resources to one high-priority task, but it can allocate resources to multiple low-priority tasks.

[0096] Furthermore, the implementation of task priority levels will be explained:

[0097] Two PriorityClasses are created in the server node using a YAML file, and their PriorityClassNames are set to HighPriority and LowPriority respectively, representing high-priority and low-priority tasks. Users can specify the priority level of a task pod using the PriorityClassName field in the YAML file when creating the task pod.

[0098] Optionally, in the resource allocation method, after determining whether the priority level of the first task is the first priority, the method further includes:

[0099] If the number of tasks in the waiting queue is greater than one, and if the priority level of the first task is not the first priority, and there is no task with the first priority level in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks; the second task is at least one task in the waiting queue other than the first task.

[0100] In this embodiment of the invention, when the number of tasks in the waiting queue is greater than one, the priority level of the first task is not the first priority, and at least one task in the waiting queue other than the first task, i.e., the priority level of the second task, is also not the first priority, the first GPU resource utilization rate of the first task is less than the first GPU limit, and the first GPU resource utilization rate of the first task is less than or equal to the first GPU resource utilization rate of the second task, the token scheduling module can determine the resource authorization information, i.e., the token, according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks, and send the token to the first container.

[0101] It should be noted that when the number of tasks in the waiting queue is greater than one, the priority level of the first task is not the first priority, and the priority level of at least one task in the waiting queue other than the first task, i.e., the second task, is also not the first priority, i.e., all tasks in the waiting queue with more than one priority are of the second priority, i.e., low priority, the token scheduling module will, under the premise of satisfying the first GPU computing unit requirements of all tasks in the waiting queue, evenly allocate the remaining resources of the first GPU, thereby improving GPU resource utilization and reducing system overhead.

[0102] It should also be noted that the first GPU resource utilization rate of the first task can be determined based on the resource utilization rate of the first GPU in the history of the first task running; the first GPU limit is a preset value, based on the design configuration.

[0103] Optionally, in the resource allocation method, after determining whether the priority level of the first task is the first priority, the method further includes:

[0104] If the number of tasks in the waiting queue is one, and if the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than the first GPU limit, the resource authorization information is determined according to the number of tasks currently running on the first GPU and the priority level of the tasks.

[0105] In this embodiment of the invention, when the number of tasks in the waiting queue is one, the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than that of the first GPU, the token scheduling module can determine the resource authorization information, i.e., the token, based on the number of tasks currently running on the first GPU and the priority level corresponding to the tasks, and send the token to the first container.

[0106] In summary, when at least one task uses the same GPU simultaneously, the GPU resource usage of the first priority task is guaranteed first. Only during the gap when the first priority task is not using GPU resources can GPU resources be allocated to the second priority task. This ensures that the GPU resource usage of the first priority task is not affected by the second priority task, and makes the training speed of the first priority task as close as possible to that when it exclusively uses a GPU.

[0107] Furthermore, while ensuring the performance of the first priority task, the second priority task can utilize the gaps in GPU resources for memory and computation, thus making fuller use of GPU resources and improving GPU resource utilization.

[0108] Optionally, in the resource allocation method, determining the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks includes:

[0109] The time quota is determined according to the following formula:

[0110] T = (1 - k1n1 - k2(n2-1))T max ;

[0111] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0112] In this embodiment of the invention, the time quota can be dynamically and flexibly adjusted according to the number of tasks currently running on the first GPU and the priority level of the tasks, thereby reducing performance overhead and ensuring control granularity.

[0113] When the number of tasks on the first GPU is small, the time quota will be increased to reduce the performance overhead caused by token switching and distribution. When there are many first-priority tasks on the first GPU, the time quota will be reduced to prevent the first-priority tasks from being affected by the long interval between the previously distributed time quotas when requesting to call the resources of the first GPU, thereby ensuring the performance of the first-priority tasks, i.e., high-priority tasks.

[0114] It should be noted that k1 and k2 are empirical values ​​or determined by experiments, such that 1-k1n1-k2(n2-1)>0; n1 is determined according to the priority level of the task currently running on the first GPU. Specifically, it depends on whether there is a first priority, i.e. a high priority task, on the first GPU. When the first GPU is currently running a high priority task, n1 is 1, and when the first GPU is not currently running a high priority task, n1 is 0.

[0115] Optionally, the resource allocation method further includes:

[0116] When it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource call request is executed, so that the first GPU runs the first task.

[0117] Optionally, in the resource allocation method, when it is determined that the first task has resource authorization information for running on the first GPU, or after determining the resource authorization information of the first task, the first GPU resource call request is executed, and after the first GPU runs the first task, the method further includes:

[0118] Obtain the duration information of the first task running on the first GPU;

[0119] If the duration information exceeds the time quota, the execution of the first GPU resource call request is stopped.

[0120] It should be noted that the resource allocation method in this embodiment of the invention can record the time when the first task is currently using the first GPU and the time when the first task starts using the first GPU resources, thereby determining the duration information of the first task using the first GPU for video memory and computation. If the duration information is greater than the time quota, the execution of the first GPU resource call request is stopped.

[0121] The following is combined with Figure 2 The resource allocation process will be explained in detail.

[0122] Step S201: The first container obtains the first GPU resource call request, that is, calls the first GPU CUDA API function;

[0123] Step S202: Before loading the CUDA dynamic link library, the first container loads an interception library to intercept the first GPU resource call request;

[0124] Step S203: Check if the first container has a token;

[0125] Step S204: If the first container has a token, determine whether the token has expired;

[0126] Step S205: If the token has not expired, execute the first GPU resource call request to allow the task to use the first GPU for video memory and computation;

[0127] Step S206: If the first container does not have a token, or the token has expired, a token request is sent to the token scheduling module;

[0128] Step S207: The token scheduling module selects a task from the waiting queue, wherein the waiting queue includes at least one task to be allocated the first GPU resource, and here, the first task is selected from the waiting queue;

[0129] Step S208: Determine whether the priority level of the first task is high priority;

[0130] Step S209: If the priority level of the first task is high priority, calculate the time quota of the first task based on the number of tasks currently running on the first GPU and the priority level of the tasks, and allocate the first GPU resources to the first task.

[0131] Step S210: If the priority level of the first task is not high priority, determine whether at least one task in the waiting queue other than the first task is high priority.

[0132] If a task in the waiting queue other than the first task is of high priority, then the allocation of the first GPU resources to the first task is stopped, and steps S207 to S209 are re-executed. The token scheduling module selects the task and calculates the time quota of the task according to the number of tasks currently running on the first GPU and the priority level of the task, and allocates the first GPU resources to the task.

[0133] Step S211: If at least one task in the waiting queue other than the first task is not a high priority, determine whether the first GPU resource utilization rate of the first task is less than the first GPU limit.

[0134] If the first GPU resource utilization rate of the first task is greater than or equal to the first GPU limit, then stop allocating the first GPU resource to the first task and re-execute step S207. The token scheduling module selects a task other than the first task from the waiting queue and allocates the first GPU resource to it.

[0135] Step S212: If the first GPU resource utilization rate of the first task is less than the first GPU limit, determine whether the first GPU resource utilization rate of the first task is less than the first GPU resource utilization rate of all tasks in the waiting queue except the first task, that is, whether the first GPU resource utilization rate of the first task is the minimum.

[0136] If the first GPU resource utilization rate of the first task is not the minimum, then step S207 is repeated, and the token scheduling module selects a task other than the first task from the waiting queue and allocates the first GPU resource to it.

[0137] If the first GPU resource utilization rate of the first task is the minimum, then step S209 is executed: calculate the time quota of the first task based on the number of tasks currently running on the first GPU and the priority level of the tasks, and allocate the first GPU resources to the first task.

[0138] Step S213: Once the allocation of the first GPU resources for the first task is completed, remove the first task from the waiting queue;

[0139] Step S214: Return the time quota to the first task and the token to the first container to complete the resource allocation.

[0140] like Figure 3 As shown, this embodiment of the invention also provides an electronic device 300, including a processor 310 and a transceiver, wherein:

[0141] The processor 310 is used to determine whether the first task has resource authorization information running on the first GPU when it receives the first GPU resource call request of the first task.

[0142] The processor 310 is further configured to, when determining that the first task does not have resource authorization information running on the first GPU, determine the resource authorization information of the first task based on the number of tasks in the waiting queue, the priority level and graphics processor resource utilization of each task in the waiting queue, and the number of tasks currently running on the first GPU and the priority level of each task.

[0143] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0144] In this embodiment of the invention, when a first graphics processor (GPU) resource request for a first task is received, it is determined whether the first task has resource authorization information running on the first GPU. If it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks. This can ensure the performance of the first priority task, improve GPU resource utilization, and reduce system overhead.

[0145] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0146] If the number of tasks in the waiting queue is at least one, determine whether the priority level of the first task is the first priority.

[0147] If the priority level of the first task is the first priority, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0148] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0149] If the number of tasks in the waiting queue is greater than one, and if the priority level of the first task is not the first priority, and there is no task with the first priority level in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks; the second task is at least one task in the waiting queue other than the first task.

[0150] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0151] If the number of tasks in the waiting queue is one, and if the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than the first GPU limit, the resource authorization information is determined according to the number of tasks currently running on the first GPU and the priority level of the tasks.

[0152] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0153] The time quota is determined according to the following formula:

[0154] T = (1 - k1n1 - k2(n2-1))T max ;

[0155] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0156] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0157] When it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource call request is executed, so that the first GPU runs the first task.

[0158] Optionally, in the electronic device 300, the processor 310 is specifically used for:

[0159] Obtain the duration information of the first task running on the first GPU;

[0160] If the duration information exceeds the time quota, the execution of the first GPU resource call request is stopped.

[0161] It should be noted that the electronic device provided in the embodiments of the present invention can implement all the method steps implemented in the above resource allocation method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0162] like Figure 4 As shown, this embodiment of the invention also provides a resource allocation device, including:

[0163] The first judgment module 410 is used to determine whether the first task has resource authorization information running on the first GPU when the first GPU resource call request of the first task is obtained;

[0164] The first determining module 420 is used to determine the resource authorization information of the first task based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks when it is determined that the first task does not have resource authorization information running on the first GPU.

[0165] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0166] In this embodiment of the invention, when a first graphics processor (GPU) resource request for a first task is received, it is determined whether the first task has resource authorization information running on the first GPU. If it is determined that the first task does not have resource authorization information running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks. This can ensure the performance of the first priority task, improve GPU resource utilization, and reduce system overhead.

[0167] Optionally, in the resource allocation device, the first determining module 420 includes:

[0168] The first judgment unit is used to determine whether the priority level of the first task is the first priority when the number of tasks in the waiting queue is at least one.

[0169] The first determining unit is configured to determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks if the priority level of the first task is the first priority.

[0170] Optionally, the resource allocation device further includes:

[0171] The second determining module is configured to, when the number of tasks in the waiting queue is greater than one, if the priority level of the first task is not the first priority, if there is no task with the priority level of the first priority in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0172] The second task is at least one task in the waiting queue other than the first task.

[0173] Optionally, the resource allocation device further includes:

[0174] The third determining module is used to determine the resource authorization information based on the number of tasks currently running on the first GPU and the priority level of the tasks when the number of tasks in the waiting queue is one, if the priority level of the first task is not the first priority, and if the first GPU resource utilization rate of the first task is less than the first GPU limit.

[0175] Optionally, in the resource allocation device, the first determining module 420 includes:

[0176] The second determining unit is used to determine the time quota according to the following formula:

[0177] T = (1 - k1n1 - k2(n2-1))T max ;

[0178] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0179] Optionally, the resource allocation device further includes:

[0180] The execution module is configured to execute the first GPU resource call request when it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, so that the first GPU runs the first task.

[0181] Optionally, the resource allocation device further includes:

[0182] The acquisition module is used to acquire information about the duration of the first task running on the first GPU;

[0183] The stop module is used to stop executing the first GPU resource call request if the duration information is greater than the time quota.

[0184] It should be noted that the apparatus provided in the embodiments of the present invention can implement all the method steps implemented in the above resource allocation method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0185] This invention also provides an electronic device, such as... Figure 5 As shown, it includes: a processor 501; and a memory 503 connected to the processor 501 via a bus interface 502. The memory 503 is used to store programs and data used by the processor 501 when performing operations. The processor 501 calls and executes the programs and data stored in the memory 503.

[0186] The transceiver 504 is connected to the bus interface 502 and is used to receive and send data under the control of the processor 501. Specifically, the processor 501 executes the following procedures:

[0187] When a first graphics processor (GPU) resource request is received for the first task, it is determined whether the first task has resource authorization information running on the first GPU.

[0188] When it is determined that the first task does not have resource authorization information for running on the first GPU, the resource authorization information of the first task is determined based on the number of tasks in the waiting queue, the priority level of each task in the waiting queue, the resource utilization rate of the first GPU, and the number of tasks currently running on the first GPU and the priority level of the tasks.

[0189] The waiting queue includes at least one task awaiting allocation of first GPU resources; the resource authorization information includes the time quota for the first task to run on the first GPU.

[0190] Optionally, the processor 501 is specifically used for:

[0191] If the number of tasks in the waiting queue is at least one, determine whether the priority level of the first task is the first priority.

[0192] If the priority level of the first task is the first priority, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks.

[0193] Optionally, the processor 501 is specifically used for:

[0194] If the number of tasks in the waiting queue is greater than one, and if the priority level of the first task is not the first priority, and there is no task with the first priority level in the waiting queue, and the first GPU resource utilization rate of the first task is less than the first GPU limit, and if the first GPU resource utilization rate of the first task in the waiting queue is less than or equal to the first GPU resource utilization rate of the second task, the resource authorization information is determined based on the number of tasks currently running on the first GPU and the priority level of the tasks; the second task is at least one task in the waiting queue other than the first task.

[0195] Optionally, the processor 501 is specifically used for:

[0196] If the number of tasks in the waiting queue is one, and if the priority level of the first task is not the first priority, and the first GPU resource utilization rate of the first task is less than the first GPU limit, the resource authorization information is determined according to the number of tasks currently running on the first GPU and the priority level of the tasks.

[0197] Optionally, the processor 501 is specifically used for:

[0198] The time quota is determined according to the following formula:

[0199] T = (1 - k1n1 - k2(n2-1))T max ;

[0200] Where T is the time quota; T max n1 is the maximum time quota of the first GPU; n2 is the number of tasks currently running on the first GPU; k1 is the first preset coefficient; and k2 is the second preset coefficient.

[0201] Optionally, the processor 501 is specifically used for:

[0202] When it is determined that the first task has resource authorization information to run on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource call request is executed, so that the first GPU runs the first task.

[0203] Optionally, the processor 501 is specifically used for:

[0204] Obtain the duration information of the first task running on the first GPU;

[0205] If the duration information exceeds the time quota, the execution of the first GPU resource call request is stopped.

[0206] Among them, Figure 5 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 501) and memory (memory 503). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 504 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. Processor 501 is responsible for managing the bus architecture and general processing, and memory 503 can store data used by processor 501 during operation.

[0207] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing the relevant hardware to implement them. The program includes instructions to perform some or all of the steps of the above methods; and the program can be stored in a readable storage medium, which can be any form of storage medium.

[0208] This invention also provides a readable storage medium, wherein a program is stored on the readable storage medium, and when the program is executed by a processor, it implements the resource allocation method as described in any of the preceding claims.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0210] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0211] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions that cause a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the transmission and reception methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] The above describes the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A resource allocation method, characterized by, The method comprises: when a first graphics processing unit (GPU) resource call request of a first task is acquired, determining whether the first task has resource authorization information running on a first GPU; when it is determined that the first task does not have resource authorization information running on the first GPU, determining resource authorization information of the first task according to a number of tasks in a waiting queue, a priority level corresponding to each task in the waiting queue and a first GPU resource usage rate, and a number of tasks currently running on the first GPU and a priority level corresponding to the tasks, comprising: when the number of tasks in the waiting queue is one, if the priority level of the first task is not a first priority, and the first GPU resource usage rate of the first task is less than a first GPU quota, determining the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks. The waiting queue comprises at least one task to be allocated a first GPU resource, and the resource authorization information comprises a time quota of the first task running on the first GPU.

2. The resource allocation method of claim 1, wherein, when it is determined that the first task does not have resource authorization information running on the first GPU, determining resource authorization information of the first task according to a number of tasks in a waiting queue, a priority level corresponding to each task in the waiting queue and a first GPU resource usage rate, and a number of tasks currently running on the first GPU and a priority level corresponding to the tasks, comprising: when the number of tasks in the waiting queue is at least one, determining whether the priority level of the first task is a first priority. If the priority level of the first task is the first priority, determining the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks.

3. The resource allocation method of claim 2, wherein, After the determination of whether the priority level of the first task is the first priority, the method further comprises: when the number of tasks in the waiting queue is more than one, if the priority level of the first task is not the first priority, and there is no task with the first priority in the waiting queue and the first GPU resource usage rate of the first task is less than a first GPU quota, if the first GPU resource usage rate of the first task is less than or equal to a first GPU resource usage rate of a second task in the waiting queue, determining the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks. The second task is at least one task in the waiting queue except the first task.

4. The resource allocation method of claim 1, wherein, Determining the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks comprises: determining the time quota according to the following formula: ； wherein, is the time quota; is the maximum time quota of the first GPU; is determined according to a priority level corresponding to a task currently running by the first GPU; is the number of tasks currently running by the first GPU; is a first preset coefficient; is a second preset coefficient.

5. The method of claim 1, wherein, The method further comprises: when it is determined that the first task has resource authorization information running on the first GPU, or after the determination of the resource authorization information of the first task, executing the first GPU resource call request, so that the first GPU runs the first task.

6. The dispensing method of claim 5, wherein, In a case where it is determined that the first task has resource authorization information running on the first GPU, or after the resource authorization information of the first task is determined, the first GPU resource calling request is executed, and after the first GPU runs the first task, the method further comprises: obtaining time length information of running of the first task on the first GPU; if the time length information is greater than the time quota, stopping execution of the first GPU resource calling request. 7.An electronic device, comprising a processor and a transceiver, and characterized in that: the processor is configured to, when a first GPU resource calling request of a first task is obtained, determine whether the first task has resource authorization information running on a first GPU; the processor is further configured to, in a case where it is determined that the first task does not have resource authorization information running on the first GPU, determine the resource authorization information of the first task according to a number of tasks in a waiting queue, a priority level corresponding to each task in the waiting queue and a GPU resource usage rate, and a number of tasks currently running on the first GPU and a priority level corresponding to the tasks; the processor is specifically configured to, in a case where the number of tasks in the waiting queue is one, if the priority level of the first task is not a first priority, and in a case where the first GPU resource usage rate of the first task is less than a first GPU limit, determine the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks. The waiting queue comprises at least one task to be allocated a first GPU resource; and the resource authorization information comprises a time quota of running of the first task on the first GPU.

8. A resource scheduling apparatus, characterized by comprising: comprises: a first determination module configured to, when a first GPU resource calling request of a first task is obtained, determine whether the first task has resource authorization information running on a first GPU; a first determination module configured to, in a case where it is determined that the first task does not have resource authorization information running on the first GPU, determine the resource authorization information of the first task according to a number of tasks in a waiting queue, a priority level corresponding to each task in the waiting queue and a first GPU resource usage rate, and a number of tasks currently running on the first GPU and a priority level corresponding to the tasks; a third determination module configured to, in a case where the number of tasks in the waiting queue is one, if the priority level of the first task is not a first priority, and in a case where the first GPU resource usage rate of the first task is less than a first GPU limit, determine the resource authorization information according to the number of tasks currently running on the first GPU and the priority level corresponding to the tasks. The waiting queue comprises at least one task to be allocated a first GPU resource; and the resource authorization information comprises a time quota of running of the first task on the first GPU.

9. An electronic device comprising: A transceiver, a processor, a memory, and a program or instructions stored on the memory and executable on the processor; wherein the processor implements the resource allocation method according to any one of claims 1 to 6 when executing the program or instructions.

10. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or instructions, when executed by the processor, implement the resource allocation method according to any one of claims 1 to 6.

Images