A method and apparatus for dynamically allocating GPU computing power for real-time tasks

Abstract

This invention discloses a method and apparatus for dynamically allocating GPU computing power for real-time tasks, comprising: acquiring priority information, preset execution deadline information, and execution parameter information of each parallel task currently being executed by the GPU; determining the current remaining computing power value of the GPU; inputting the execution parameter information and the sub-computing power value into a time prediction model to obtain the remaining execution time required to complete each of the parallel tasks; determining a second computing power value required by the GPU to complete the parallel tasks; and dynamically allocating computing power values ​​to each of the parallel tasks based on a comparison between the second computing power value of each parallel task and the remaining computing power value, as well as the priority information. This method can dynamically adjust quotas for applications to meet real-time requirements and ensure concurrent execution of multiple tasks sharing the GPU.

Description

Technical Field

[0001] This invention relates to the field of graphics processor computing technology, and more specifically to a method and apparatus for dynamically allocating computing power for real-time GPU tasks. Background Technology

[0002] Currently, military and civilian aircraft worldwide utilize a large number of onboard computers. These specialized devices are designed to perform only one or a few specific tasks. With highly integrated software and hardware, hundreds of onboard computers are distributed throughout the fuselage, tightly interconnected via onboard networks to form various onboard systems, ensuring flight safety and the achievement of critical missions. Onboard computers must be able to withstand harsh environments. They are required to maintain stable operation under various temperatures, humidity levels, severe vibrations, corrosive environments, and electromagnetic radiation. Furthermore, onboard computers must possess high security, strong real-time performance, and high reliability. Real-time systems, like other computing systems, must not only meet the requirements of logical correctness but also the requirements of temporal correctness (i.e., real-time performance). Real-time systems can be categorized into soft real-time systems and hard real-time systems based on their real-time performance requirements. If a task's failure to complete before the deadline only degrades the system's performance, it is a soft real-time system; if the consequences are catastrophic, it is a hard real-time system. Onboard computers belong to the hard real-time system category. Compared to general-purpose computing systems, airborne embedded systems typically have a fixed set of applications and strict real-time constraints.

[0003] Today, Graphics Processing Units (GPUs) have become indispensable general-purpose hardware accelerators, widely used in various data-parallel and computationally intensive applications. With the widespread adoption of GPUs across various fields, most advanced airborne embedded systems have also begun to use GPUs as accelerators to improve processing performance. The primary purpose of traditional GPUs is to accelerate high-performance computing applications; however, they lack a key technology for real-time systems: preemptive scheduling. When a low-priority application requests a GPU request, the inherent scheduler will not efficiently handle this request. Because the default GPU scheduling does not consider application priority, a higher-priority task arriving later may be blocked or deleted by the lower-priority task due to resource consumption, resulting in a priority inversion problem. With the widespread application of GPUs in highly critical embedded systems such as autonomous driving technology and airborne embedded systems, research into GPU real-time scheduling technology is both necessary and meaningful. GPUs, managed by the operating system as coprocessor units for acceleration, are subject to real-time performance limitations. Since GPUs mostly operate in a non-preemptive mode (meaning they cannot be interrupted until the current workload is complete), if a low-priority application task occupies GPU resources, a high-priority application task may miss its deadline due to being unable to use the GPU. Therefore, for such system scheduling goals, ensuring real-time performance must be the top priority. However, current general GPU scheduling goals typically focus on improving task execution efficiency and throughput, without considering the real-time performance of application task scheduling. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, a first aspect of this invention proposes a method for dynamically allocating computing power for real-time GPU tasks, the method comprising:

[0005] Obtain priority information, preset execution deadline information, and execution parameter information for each parallel task currently being executed by the GPU. The execution parameter information includes: GPU resource utilization, memory utilization, GPU performance, GPU power, GPU temperature, core frequency, memory frequency, utilization rate, memory efficiency, computing power, and task execution time.

[0006] Determine the current remaining computing power of the GPU based on the first computing power value currently allocated to each parallel task;

[0007] By inputting the execution parameter information and sub-computing power value into the time prediction model, the remaining execution time required to complete each parallel task is obtained;

[0008] Based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task, determine the second computing power value required for the GPU to complete the parallel task;

[0009] Based on the comparison results of the second computing power value and the remaining computing power value of each parallel task, as well as the priority information, the computing power value allocated to each parallel task is dynamically adjusted.

[0010] Preferably, the remaining computing power of the GPU is determined based on the first computing power value currently allocated to each parallel task, including:

[0011] Get the total computing power of the GPU;

[0012] Determine the sum of the first computing power values ​​currently allocated to each parallel task by the GPU;

[0013] The remaining computing power of the GPU is determined by the difference between the total computing power of the GPU and the sum of the first computing power.

[0014] Preferably, obtaining the priority information and preset execution deadline information of each parallel task currently being executed by the GPU also includes:

[0015] Retrieve the unified computing device architecture request when the highest priority parallel task on the running GPU starts;

[0016] Based on the request from the unified computing device architecture, the corresponding Pod is configured for the highest priority parallel task, where Pod represents the smallest scheduling unit in Kubernetes.

[0017] Preferably, the second computing power value required for the GPU to complete the parallel task is determined based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task, including:

[0018] Obtain multiple different sub-computing power values ​​for any one of the multiple parallel tasks;

[0019] Sort the multiple first computing power values ​​from low to high;

[0020] Input multiple sub-computing power values ​​sorted from smallest to largest into the time prediction model, and output multiple corresponding first-ranked task execution times;

[0021] The first task execution time is the smallest first task execution time whose first quotient of the first task execution time and the preset execution deadline is greater than 1.

[0022] The sub-computing power value is determined based on the execution time of the second task;

[0023] Based on the sub-computing power value, determine the second computing power value required to execute each parallel task.

[0024] Preferably, based on the comparison results of the second computing power value and the remaining computing power value of each parallel task, as well as priority information, the computing power value allocated to each parallel task is dynamically adjusted, including:

[0025] The second quotient of the second computing power value required to obtain the highest priority task and the first remaining computing power value of the GPU;

[0026] If the second quotient is less than 1, then according to the second computing power value required by the highest priority task, the first remaining computing power value is allocated to the highest priority task, and the second remaining computing power value is obtained.

[0027] Allocate the remaining second computing power value proportionally to the currently running tasks;

[0028] When the second highest priority task arrives, obtain the second quotient of the second highest priority task's required second computing power value and the GPU's second remaining computing power value;

[0029] If the second quotient is less than 1, then the second remaining computing power value is allocated according to the second computing power value required by the task with the second highest priority.

[0030] If the second quotient is greater than 1, the scheduling fails;

[0031] Repeat the above steps until all parallel task computing power is allocated.

[0032] Preferably, after allocating the first remaining computing power value to the highest priority task, the method further includes:

[0033] Initialize the Pod based on the second-highest computing power required by the highest-priority task;

[0034] Execute the highest priority task in the Pod.

[0035] The present invention also provides a device for dynamically allocating GPU real-time task computing power, comprising:

[0036] The first acquisition module is used to acquire priority information, preset execution deadline information, and execution parameter information of each parallel task currently being executed by the GPU. The execution parameter information includes: GPU resource utilization, memory utilization, GPU performance, GPU power, GPU temperature, core frequency, memory frequency, utilization rate, memory efficiency, computing power value, and task execution time.

[0037] The second acquisition module is used to determine the current remaining computing power of the GPU based on the first computing power value currently allocated to each parallel task by the GPU.

[0038] The time prediction module is used to input execution parameter information and sub-computing power values ​​into the time prediction model to obtain the remaining execution time required to complete each parallel task;

[0039] The computing power determination module is used to determine the second computing power value required for the GPU to complete the parallel task based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task.

[0040] The computing power allocation module is used to dynamically allocate computing power to each parallel task based on the comparison results of the second computing power value and the remaining computing power value of each parallel task, as well as priority information.

[0041] The present invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to realize the GPU real-time task computing power dynamic allocation method as described in the first aspect.

[0042] The present invention also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the GPU real-time task computing power dynamic allocation method as described in the first aspect.

[0043] This invention provides a method and apparatus for dynamically allocating computing power for real-time GPU tasks. Compared with the prior art, its advantages are as follows:

[0044] 1. Sharing GPUs in Kubernetes not only improves GPU utilization but also provides excellent sharing capabilities and additional advantages such as dynamic migration, without affecting application operation.

[0045] 2. This method uses a backpropagation artificial neural network to implement a GPU performance model and predict the execution time of GPU tasks.

[0046] 3. The real-time scheduling method based on dynamic quota implemented in this paper can dynamically adjust the quota for the application to meet the real-time requirements. The scheduling strategy is flexible, and it can achieve near preemption of multiple tasks, which can ensure that multiple tasks share the GPU for concurrent execution. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0048] Figure 1A flowchart of a method for dynamically allocating GPU computing power for real-time tasks provided in an embodiment of the present invention;

[0049] Figure 2 This is a diagram illustrating a method for dynamically allocating GPU computing power for real-time tasks in Kubernetes, as provided in an embodiment of the present invention.

[0050] Figure 3 This invention provides an overall structural diagram of a GPU virtualization-based scheduling framework for a method of dynamically allocating computing power for real-time GPU tasks.

[0051] Figure 4 This is a time prediction model structure diagram of a method for dynamically allocating computing power for real-time GPU tasks provided in an embodiment of the present invention;

[0052] Figure 5 This is a flowchart of a task soft real-time scheduling method for a GPU real-time task computing power dynamic allocation method provided in an embodiment of the present invention;

[0053] Figure 6 This is a structural block diagram of a GPU real-time task computing power dynamic allocation device provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This specification provides the operational steps for the methods described in the embodiments or flowcharts, but may include more or fewer operational steps based on conventional or non-inventive labor. In actual system or server product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0056] Figure 1 A flowchart of a method for dynamically allocating GPU computing power for real-time tasks provided in an embodiment of the present invention;

[0057] like Figure 1 As shown, the preparation methods include:

[0058] Step 101: Obtain the priority information, preset execution deadline information, and execution parameter information of each parallel task currently being executed by the GPU; the execution parameter information includes: GPU resource utilization, memory utilization, GPU performance, GPU power, GPU temperature, core frequency, memory frequency, utilization rate, memory efficiency, sub-computing power value, and task execution time.

[0059] The proposed multi-GPU real-time task scheduling mechanism based on dynamic quotas is logically divided into a GPU virtualization layer and a GPU scheduling layer. The virtualization layer consists of several parts: Kubelet, vGPU Library, and GPU Manager.

[0060] Kubelet provides device plugins for discovering GPU resources, allocating appropriate hardware resources to tasks, and configuring the container runtime environment. A device plugin is registered with Kubelet via the gRPC service. Kubelet can then obtain device information from the device plugin and manage the device resources.

[0061] vGPU Library: GaiaGPU uses VCUDA for GPU virtualization. The vCUDA library is a whopper library that intercepts and redirects CUDA calls from applications. The vGPU Library runs in a container and manages the GPU resources of that container. It is loaded the first time a GPU program is executed within the container. The vGPU Library intercepts memory- and compute-related APIs in the CUDA library by utilizing the LD_LIBRARY_PATH mechanism (an environment variable in Linux). Directories included in LD_LIBRARY_PATH are loaded before the standard library directories.

[0062] GPU Manager is an instantiated device plugin used to publish GPU resources to Kubelet. It is responsible for creating vGPUs and reporting GPU status to Kubelet via gRPC communication. Once successfully registered with Kubelet, it returns a list of devices. GPU Manager sends vGPU information to Kubelet instead of directly returning physical GPU information. Physical GPUs are virtualized in two dimensions: memory (vmemory) and compute resources (vprocessors). Specifically, a vmemory device is a 256MB memory unit, and a vprocessor device is 1% of the GPU CUDA cores (a physical GPU card is divided into 100 vprocessors). When an application initiates a CUDA request in a container, Kubelet sends an allocation request to GPU Manager. GPU Manager then sends a scheduling request to the GPU scheduling layer. If the application is successfully allocated to a physical device, it returns an allocation response containing the configuration for accessing the allocated device.

[0063] The scheduling layer comprises a scheduler, a task remaining execution time predictor, a task quota configurator, and a GPU information manager. The scheduling layer receives scheduling requests from the GPU Manager and dynamically configures the required GPU quota for application tasks. This method redesigns the GPU scheduling layer in GaiaGPU and implements a real-time scheduling method based on dynamic quotas. It dynamically adjusts the quotas (vprocessors) for applications to meet real-time demands, rather than deleting low-priority applications at scheduling points. In the inherent scheduling of GPUs, when multiple tasks arrive simultaneously, the scheduler allocates GPUs according to a first-come, first-served principle. Therefore, it cannot guarantee that tasks will complete before their deadlines when multiple real-time tasks are scheduled. This scheduling mechanism effectively solves this problem, supporting flexible scheduling strategies to meet the aforementioned requirements while ensuring concurrent execution of multiple tasks sharing the GPU.

[0064] Step 102: Determine the current remaining computing power of the GPU based on the first computing power value currently allocated to each parallel task;

[0065] In one possible implementation, the remaining computing power of the GPU is determined based on the first computing power value currently allocated to each parallel task, specifically including:

[0066] Get the total computing power of the GPU;

[0067] Determine the sum of the first computing power values ​​currently allocated to each parallel task by the GPU;

[0068] The remaining computing power of the GPU is determined by the difference between the total computing power of the GPU and the sum of the first computing power.

[0069] In one possible implementation, determining the first remaining computing power value of the GPU based on the difference between the total computing power value of the GPU and the currently occupied computing power value includes:

[0070] Get the first list of parallel tasks in each Pod;

[0071] Get a second list of parallel tasks existing in the GPU memory;

[0072] Pods that do not exist in the first list but exist in the second list are considered dead Pods;

[0073] Release the GPU computing power occupied by the destroyed Pod;

[0074] Pods that exist in the first list but not in the second list will be added to the GPU memory as surviving Pods.

[0075] Periodically check the GPU computing power allocated to surviving Pods;

[0076] The first remaining computing power of the GPU is determined by the difference between the total computing power of the GPU and the sum of the computing power of the GPU allocated to each surviving Pod.

[0077] In the embodiments provided by this invention, the GPU Information Manager is used to maintain a GPU resource record table and periodically statistically analyze the program status of the current application tasks. It calculates the GPUs used by each application task and their corresponding quotas, and then calculates the remaining GPU quota based on the statistical data. Simultaneously, it periodically monitors whether the currently executing task containers are still alive. If a task container completes execution and exits, the task is deleted from the GPU resource table, the resource list is updated, and the quota is returned. It is responsible for recording all task-related information, such as task arrival time, task scheduling time, and task end time. During queries, the task name is used as the sole criterion for searching. First, the GPU Information Manager needs to initialize objects, allocate space for the data structure storing task information, and has the function of periodically querying and persisting task information. Then, it is responsible for recording the times of events occurring throughout the entire lifecycle of the task to a log file. When the task execution cycle ends, the Task Information Manager saves pod information such as arrival time, start scheduling time, and completion time to the log file in sequence, with the time unit calculated in nanoseconds, but ultimately converted to seconds before unified storage. Secondly, the Task Information Manager also needs to detect the current status of the task, whether it is currently executing or has already completed execution. Based on the list of currently alive pods, and comparing it with the pods recorded in memory, newly arriving tasks and completed tasks are retrieved. After traversing the pod list in the task set, if a pod is not found in the list of currently alive pods, it is considered dead, and the time of the pod's death event is recorded. If a currently alive pod is not found in the list of pods in memory, its information is added to memory. Simultaneously, a periodic detection task is performed: periodically calling the task information monitoring function to detect and record pod death times, and periodically executing a task to iterate through the GPU quotas allocated to alive pods to calculate the remaining quotas for each GPU on the current node. The GPU information manager also needs to periodically write data to the log file. The GPU information manager runs a separate coroutine that iterates through the task information data in the task set every period. If all elements in the time array corresponding to a task in the task set are not 0, it means that the current execution lifecycle of this task has ended. At this time, the task's information is output to the log file, and the task is deleted from the task set.

[0078] Step 103: Input the execution parameter information and sub-computing power value into the time prediction model to obtain the remaining execution time required to complete each parallel task;

[0079] Specifically, firstly, the scheduling framework abstracts and divides GPU hardware resources into two dimensions: storage resources and computing resources, and intercepts CUDA access requests from application tasks. Then, by analyzing the runtime computing resource status, it achieves scheduling management of multiple real-time GPU tasks on a single GPU. The scheduler works by dynamically configuring the GPU quota used for task execution according to a certain scheduling policy, based on whether the remaining execution time of arriving tasks exceeds the deadline. Executed tasks are not removed from the GPU, and low-priority tasks with longer deadlines are assigned smaller quotas. Scheduling is required when a new task arrives or a task ends. During each scheduling, a task remaining execution time predictor is used to predict the remaining execution time of the task, which is then compared with the task's preset execution deadline. Tasks exceeding the deadline require adjustment of their GPU quota. After determining the quota, the task quota configurator is used to configure the GPU quota for the task. Simultaneously, the GPU resource record table in the GPU information manager is updated. During application task execution, GPU load-related metrics are obtained via the API provided by nvprof as input, and the actual measured GPU task execution time is used as output. In each iteration, the weights on the connections are updated using the error between the predicted and actual task execution times. These model parameters, based on a neural network, predict the remaining execution time of the application. For each scheduling, a task remaining execution time predictor is used to forecast the remaining time of the currently executing load, and the minimum GPU quota required for the task is calculated based on the prediction results.

[0080] Step 104: Determine the second computing power value required for the GPU to complete the parallel task based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task;

[0081] In one possible implementation, a second computing power value required for the GPU to complete the parallel task is determined based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task, including:

[0082] Obtain multiple different sub-computing power values ​​for any one of the multiple parallel tasks;

[0083] Sort the multiple sub-computing power values ​​from low to high;

[0084] Input multiple sub-computing power values ​​sorted from smallest to largest into the time prediction model, and output multiple corresponding first-ranked task execution times;

[0085] The first task execution time is the smallest first task execution time whose first quotient of the first task execution time and the preset execution deadline is greater than 1.

[0086] The minimum sub-computing power value is determined based on the execution time of the second task;

[0087] Based on the lowest sub-computing power value, determine the second computing power value required to execute each parallel task.

[0088] In the embodiments provided by the present invention, multiple sub-computing power values ​​sorted from smallest to largest are input into the time prediction model, and multiple corresponding sorted first task execution times are output. If the first task execution time sorted from smallest to largest is greater than the sub-computing power value corresponding to the preset execution deadline for the first time, it is taken as the lowest sub-computing power value, and the lowest sub-computing power value is taken as the second computing power value.

[0089] Step 105: Based on the comparison results of the second computing power value and the remaining computing power value of each parallel task, as well as the priority information, dynamically allocate the computing power value to each parallel task.

[0090] In one possible implementation, the computing power allocated to each parallel task is dynamically adjusted based on the comparison result between the second computing power value and the remaining computing power value of each parallel task, as well as priority information, including:

[0091] The second quotient of the second computing power value required to obtain the highest priority task and the first remaining computing power value of the GPU;

[0092] If the second quotient is less than 1, then according to the second computing power value required by the highest priority task, the first remaining computing power value is allocated to the highest priority task, and the second remaining computing power value is obtained.

[0093] Allocate the remaining second computing power value proportionally to the currently running tasks;

[0094] When the second highest priority task arrives, obtain the second quotient of the second highest priority task's required second computing power value and the GPU's second remaining computing power value;

[0095] If the second quotient is less than 1, then the second remaining computing power value is allocated according to the second computing power value required by the task with the second highest priority.

[0096] If the second quotient is greater than 1, the scheduling fails;

[0097] Repeat the above steps until all parallel task computing power is allocated.

[0098] In summary, this method proposes a task execution time prediction model. In complex environments involving task execution time and multi-task parallel execution, numerous parameters influence application task execution time. By analyzing the relationship between task execution time and computational resources, parameters related to GPU resources and quotas are selected as model parameters. Using nvprof, parameters related to task execution time are collected during application task runtime. Since airborne embedded systems generally require real-time performance, task execution time prediction is necessary. Therefore, the collected data is used to train a neural network to predict task execution time, enabling more accurate prediction of the execution time of a specific application task under multi-application task parallelism. This allows for the estimation of the remaining execution time of the application task, and a dynamic quota adjustment algorithm is proposed. Based on the task execution time, the remaining execution time is estimated, along with factors such as deadlines and priorities, to dynamically adjust task quotas, allocating quotas that allow tasks to complete before the deadline. Based on the predicted quotas, application task scheduling is implemented using a soft real-time scheduling method.

[0099] Figure 2 This diagram illustrates a method for dynamically allocating GPU computing power for real-time tasks in Kubernetes, as provided in an embodiment of the present invention.

[0100] like Figure 2 As shown, based on GPU-related virtualization technologies and combined with Kubernetes container orchestration technology, a physical GPU is first partitioned into vGPUs within a virtual machine using virtualization drivers. Each vGPU appears as a complete GPU. This allows the computing resources on the physical GPU to be divided into multiple vGPUs, which are then mapped to different virtual machines. The GPUs are then shared within Kubernetes, and Pods execute workloads within Kubernetes without requiring modifications to the task source code.

[0101] Among them, Kubernetes (abbreviated as K8s, where 8 refers to the middle 8 characters) is a brand-new distributed architecture solution based on container technology. Although this solution is still very new, it is an important result of Google's experience in the large-scale application of container technology over the past ten years. To be precise, it is an open-source version of Borg, a well-known large-scale cluster management system used internally by Google. Its purpose is to automate resource management and maximize resource utilization across data centers.

[0102] Kubernetes has comprehensive cluster management capabilities, including multi-layered security protection and access control mechanisms, multi-tenant application support capabilities, transparent service registration and service discovery mechanisms, built-in intelligent load balancers, powerful fault detection and self-healing capabilities, rolling service upgrades and online scaling capabilities, scalable automatic resource scheduling mechanisms, and robust resource quota management capabilities.

[0103] Meanwhile, Kubernetes provides comprehensive management tools that cover all aspects, including development, deployment, testing, operation and maintenance monitoring. It is not only a brand-new distributed architecture solution based on container technology, but also a one-stop complete distributed system development and support platform.

[0104] Figure 3 The diagram shows the overall structure of a GPU virtualization-based scheduling framework for a method of dynamically allocating computing power for real-time GPU tasks, as provided in an embodiment of the present invention.

[0105] like Figure 3 As shown, the dynamic allocation method for GPU real-time task computing power is logically divided into a GPU virtualization layer and a GPU scheduling layer. First, the scheduling framework abstracts and divides GPU hardware resources from two dimensions: storage resources and computing resources, and intercepts CUDA access requests from application tasks. Then, by analyzing the runtime computing resource status, it achieves scheduling management of multiple GPU real-time tasks on a single GPU. The scheduler works by dynamically configuring the GPU quota used for task execution according to a certain scheduling strategy, based on whether the remaining execution time of arriving tasks exceeds the deadline. Executed tasks are not removed from the GPU, and low-priority tasks with longer deadlines are allocated smaller quotas. Scheduling is required when a new task arrives or a task ends. During each scheduling, a task remaining execution time predictor is used to predict the remaining execution time of the task, which is then compared with the task's relative deadline. Tasks exceeding the deadline require adjustment of their GPU quota. After determining the quota, a task quota configurator is used to configure the GPU quota for the task. Simultaneously, the GPU resource record table in the GPU information manager is updated.

[0106] Figure 4 This is a network structure diagram of a GPU performance model for a method of dynamically allocating computing power for real-time GPU tasks, provided in an embodiment of the present invention.

[0107] like Figure 4 As shown, a time prediction model has been added to the GaiaGPU framework. This model uses a configurable backpropagation artificial neural network (BP-ANN) to predict the execution time of application tasks based on parameters related to task execution time and the task's GPU quota usage. Figure 3As shown, a four-layer network is used. Since the task set in embedded systems is generally fixed, each GPU task corresponds to its own artificial neural network model. The neural network-based task execution time prediction model uses 20 carefully selected parameters related to task execution time as inputs to the input layers. Based on experiments, two inner hidden layers are used, each with 8 neurons for training input. The output layer has only one neuron, which is the output of the GPU task execution time prediction. Each layer is tightly connected to the previous and next layers through specific weight values. During the training phase, when training the artificial neural network model for a GPU task, other tasks are randomly combined to form the workload of that task. During the application task execution, GPU load-related metrics are obtained through the API provided by nvprof as input, and the actual measured GPU task execution time is used as output. Each iteration uses the error between the task execution time prediction and the actual task execution time to update the weights on the connections. Using these model parameters and based on the neural network, the remaining execution time of the executed application is predicted.

[0108] Figure 5 The flowchart of a task soft real-time scheduling method for a GPU real-time task computing power dynamic allocation method provided in an embodiment of the present invention is shown.

[0109] like Figure 5 As shown, when a task arrives, this method first obtains the task's priority. For the task with the highest priority, it first calculates the remaining execution time of the first task. Then, it determines whether to schedule the task based on the remaining execution time of the first task and the preset execution deadline. If the remaining execution time of the first task is greater than the preset execution deadline, it calculates the computing power required for the task based on the preset execution deadline. If the required computing power does not exceed the computing power provided by the system, it allocates the remaining computing power. The above steps are then performed on each task with lower priority until all tasks are completed.

[0110] In one possible implementation, before obtaining the priority information and preset execution deadline information of each parallel task currently being executed by the GPU, the method further includes:

[0111] Retrieve the unified computing device architecture request when the highest priority parallel task on the running GPU starts;

[0112] Based on the request from the unified computing device architecture, the corresponding Pod is configured for the highest priority parallel task, where Pod represents the smallest scheduling unit in Kubernetes.

[0113] In the embodiments provided by this invention, application tasks are deployed as Pods and require NVIDIA Docker as the container engine during execution. When the highest priority parallel task arrives, the GPU Manager first registers itself with the Kubelet and reports the vGPU information. The Kubelet receives a request to create a GPU container. After receiving the request, the Kubelet sends an allocation request to the GPU Manager. The GPU Manager sends a vGPU scheduling request to the scheduler. After receiving the task scheduling request, the scheduler calculates the physical GPUs configured for the Pod according to the scheduling policy.

[0114] In one possible implementation, after allocating the first remaining computing power value to the highest priority task, the method further includes:

[0115] Initialize the Pod based on the second-highest computing power required by the highest-priority task;

[0116] Execute the highest priority task in the Pod.

[0117] In the embodiments provided by this invention, after the scheduling framework is successfully deployed and started, the GPU Information Manager periodically calculates resource usage and updates and maintains the current GPU information table. Upon successful scheduling, a response containing information about the physical GPU is returned. Upon receiving the response, the GPU Manager returns the container environment variables, mount information, and GPU information to the Kubelet via the allocation response. Then, the Kubelet creates and initializes a container based on the allocation response. Subsequently, vGPULibrary registers itself and manages the GPU resources of its container. After registration, the task begins execution with the quota configured according to the scheduling policy and continues until execution is complete, at which point the system reclaims resources.

[0118] Figure 6 This is a structural block diagram of a GPU real-time task computing power dynamic allocation device provided in an embodiment of the present invention. Figure 6 As shown, the task computing power dynamic allocation device 200 includes:

[0119] The first acquisition module 201 is used to acquire priority information, preset execution deadline information, and execution parameter information of each parallel task currently being executed by the GPU. The execution parameter information includes: GPU resource utilization, memory utilization, GPU performance, GPU power, GPU temperature, core frequency, memory frequency, utilization rate, memory efficiency, computing power value, and task execution time.

[0120] The second acquisition module 202 is used to determine the current remaining computing power of the GPU based on the first computing power value currently allocated to each parallel task by the GPU.

[0121] The time prediction module 203 is used to input the execution parameter information and sub-computing power value into the time prediction model to obtain the remaining execution time required to complete each parallel task;

[0122] The computing power value determination module 204 is used to determine the second computing power value required for the GPU to complete the parallel task based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task.

[0123] The computing power allocation module 205 is used to dynamically allocate computing power values ​​to each parallel task based on the comparison results of the second computing power value and the remaining computing power value of each parallel task and the priority information.

[0124] In one possible implementation, a remaining computing power value calculation module 206 is also included, which is specifically used for:

[0125] Get the total computing power of the GPU;

[0126] Determine the sum of the first computing power values ​​currently allocated to each parallel task by the GPU;

[0127] The remaining computing power of the GPU is determined by the difference between the total computing power of the GPU and the sum of the first computing power.

[0128] In one possible implementation, a scheduling request module 207 is also included, which is specifically used for:

[0129] Retrieve the unified computing device architecture request when the highest priority parallel task on the running GPU starts;

[0130] Based on the request from the unified computing device architecture, the corresponding Pod is configured for the highest priority parallel task, where Pod represents the smallest scheduling unit in Kubernetes.

[0131] In one possible implementation, the computing power value determination module 204 is specifically used for:

[0132] Obtain multiple different sub-computing power values ​​for any one of the multiple parallel tasks;

[0133] Sort the multiple first computing power values ​​from low to high;

[0134] Input multiple sub-computing power values ​​sorted from smallest to largest into the time prediction model, and output multiple corresponding first-ranked task execution times;

[0135] The first task execution time is the smallest first task execution time whose first quotient of the first task execution time and the preset execution deadline is greater than 1.

[0136] The sub-computing power value is determined based on the execution time of the second task;

[0137] Based on the sub-computing power value, determine the second computing power value required to execute each parallel task.

[0138] In one possible implementation, the computing power allocation module 205 is specifically used for:

[0139] The second quotient of the second computing power value required to obtain the highest priority task and the first remaining computing power value of the GPU;

[0140] If the second quotient is less than 1, then according to the second computing power value required by the highest priority task, the first remaining computing power value is allocated to the highest priority task, and the second remaining computing power value is obtained.

[0141] Allocate the remaining second computing power value proportionally to the currently running tasks;

[0142] When the second highest priority task arrives, obtain the second quotient of the second highest priority task's required second computing power value and the GPU's second remaining computing power value;

[0143] If the second quotient is less than 1, then the second remaining computing power value is allocated according to the second computing power value required by the task with the second highest priority.

[0144] If the second quotient is greater than 1, the scheduling fails;

[0145] Repeat the above steps until all parallel task computing power is allocated.

[0146] In one possible implementation, an execution module 208 is further included, specifically for:

[0147] Initialize the Pod based on the second-highest computing power required by the highest-priority task;

[0148] Execute the highest priority task in the Pod.

[0149] In another embodiment of the present invention, a device is also provided, the device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the point cloud data visualization processing method described in the embodiment of the present invention.

[0150] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the point cloud data visualization processing method described in the embodiment of the present invention.

[0151] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0152] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. The focus of each embodiment is on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0154] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for dynamically allocating GPU computing power for real-time tasks, characterized in that, include: Obtain priority information, preset execution deadline information, and execution parameter information for each parallel task currently being executed by the GPU; The execution parameter information includes: GPU resource utilization, video memory utilization, GPU performance, GPU power, GPU temperature, core frequency, video memory frequency, utilization rate, memory efficiency, multiple sub-computing power values, and task execution time. The remaining computing power of the GPU is determined based on the first computing power value currently allocated to each of the parallel tasks. The execution parameter information and the multiple sub-computing power values ​​are input into the time prediction model to obtain the multiple remaining execution times required to complete each of the parallel tasks; Based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task, determine the second computing power value required for the GPU to complete the parallel task; Based on the comparison result between the second computing power value of each parallel task and the remaining computing power value, as well as the priority information, the computing power value allocated to each parallel task is dynamically adjusted. The step of dynamically allocating computing power to each of the parallel tasks based on the comparison result between the second computing power value of each of the parallel tasks and the remaining computing power value, and the priority information, includes: The second quotient of the second computing power value required to obtain the highest priority task and the first remaining computing power value of the GPU; If the second quotient is less than 1, then according to the second computing power value required by the highest priority task, the first remaining computing power value is allocated to the highest priority task to obtain the second remaining computing power value; Allocate the remaining second computing power value proportionally to the currently running tasks; When the second highest priority task arrives, obtain the second quotient of the second computing power value required for the second highest priority task and the second remaining computing power value of the GPU. If the second quotient is less than 1, then the second remaining computing power value is allocated according to the second computing power value required by the task with the second highest priority. If the second quotient is greater than 1, the scheduling fails; Repeat the above until the computing power allocation for all the parallel tasks is completed.

2. The method as described in claim 1, characterized in that, Determining the current remaining computing power of the GPU based on the first computing power value currently allocated to each of the parallel tasks includes: Get the total computing power of the GPU; Determine the sum of the first computing power values ​​currently allocated by the GPU to each of the parallel tasks; The remaining computing power of the GPU is determined based on the difference between the total computing power of the GPU and the sum of the first computing power.

3. The method as described in claim 1, characterized in that, Before obtaining the priority information and preset execution deadline information of each parallel task currently being executed by the GPU, the process also includes: Obtain the unified computing device architecture request when the highest priority parallel task on the GPU starts; Based on the request from the unified computing device architecture, the corresponding Pod is configured for the highest priority parallel task, where a Pod represents the smallest scheduling unit in Kubernetes.

4. The method as described in claim 1, characterized in that, The step of determining the second computing power value required for the GPU to complete the parallel task based on the remaining execution time of the parallel task and the preset execution deadline corresponding to the parallel task includes: Obtain multiple different sub-computing power values ​​for any one of the multiple parallel tasks; Sort the multiple sub-computing power values ​​from low to high; The time prediction model inputs multiple different sub-computing power values ​​sorted from smallest to largest and outputs multiple corresponding sorted first task execution times. The first task execution time is the smallest first task execution time among those whose first quotient of the first task execution time and the preset execution deadline is greater than 1. The minimum sub-computing power value is determined based on the execution time of the second task; Based on the lowest sub-computing power value, a second computing power value required to execute each of the parallel tasks is determined.

5. The method as described in claim 1, characterized in that, After allocating the first remaining computing power value to the highest priority task, the process also includes: Initialize the Pod based on the second-highest computing power required by the highest-priority task; Execute the highest priority task in the Pod.

6. The method as described in claim 2, characterized in that, Determining the first remaining computing power value of the GPU based on the difference between the total computing power value of the GPU and the currently occupied computing power value includes: Get the first list of the parallel tasks in each Pod; Obtain a second list of the parallel tasks existing in the GPU memory; Pods that do not exist in the first list but exist in the second list are considered dead Pods; Release the GPU computing power occupied by the destroyed Pod; Pods that exist in the first list but not in the second list will be added to the GPU memory as live Pods. Periodically check the GPU computing power allocated to surviving Pods; The first remaining computing power value of the GPU is determined based on the difference between the total computing power value of the GPU and the sum of the computing power values ​​of the GPUs allocated to each surviving Pod.

7. A device for dynamically allocating GPU computing power for real-time tasks, characterized in that, include: The first acquisition module is used to acquire priority information, preset execution deadline information, and execution parameter information of each parallel task currently being executed by the GPU; The execution parameter information includes: GPU resource utilization, video memory utilization, GPU performance, GPU power, GPU temperature, core frequency, video memory frequency, utilization rate, memory efficiency, multiple sub-computing power values, and task execution time. The second acquisition module is used to determine the current remaining computing power of the GPU based on the first computing power value currently allocated to each of the parallel tasks by the GPU. The time prediction module is used to input the execution parameter information and the multiple sub-computing power values ​​into the time prediction model to obtain the multiple remaining execution times required to complete each of the parallel tasks; The computing power value determination module is used to determine the second computing power value required for the GPU to complete the parallel task based on the multiple remaining execution times of the parallel task and the preset execution deadline corresponding to the parallel task; The computing power allocation module is used to dynamically allocate computing power values ​​to each of the parallel tasks based on the comparison result between the second computing power value of each of the parallel tasks and the remaining computing power value, as well as the priority information. The computing power allocation module is specifically used for: The second quotient of the second computing power value required to obtain the highest priority task and the first remaining computing power value of the GPU; If the second quotient is less than 1, then according to the second computing power value required by the highest priority task, the first remaining computing power value is allocated to the highest priority task to obtain the second remaining computing power value; Allocate the remaining second computing power value proportionally to the currently running tasks; When the second highest priority task arrives, obtain the second quotient of the second computing power value required for the second highest priority task and the second remaining computing power value of the GPU. If the second quotient is less than 1, then the second remaining computing power value is allocated according to the second computing power value required by the task with the second highest priority. If the second quotient is greater than 1, the scheduling fails; Repeat the above until the computing power allocation for all the parallel tasks is completed.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the GPU real-time task computing power dynamic allocation method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the GPU real-time task computing power dynamic allocation method as described in any one of claims 1-6.

Images