GPU (Graphics Processing Unit) resource scheduling method and device, equipment and medium

Abstract

The invention provides a GPU resource scheduling method and device, equipment and a medium, and belongs to the technical field of data processing. The method comprises the following steps: in response to received AI tasks, obtaining equipment information of a plurality of AI tasks and a plurality of GPU equipment, the equipment information comprising bearable resource quantity and to-be-released resource quantity; analyzing each AI task by using a pre-trained task identification model to obtain a demand resource quantity of the AI task; matching each demand resource quantity with the device information of each GPU device, and determining a plurality of target GPU devices from the plurality of GPU devices; and obtaining the current task content of each target GPU device, and determining a task execution strategy according to each AI task and the current task content of each target GPU device. According to the method, AI task scheduling reasonability can be effectively improved.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a GPU resource scheduling method, apparatus, device and medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, the application scenarios of AI computing tasks are becoming increasingly widespread, covering multiple fields such as natural language processing, computer vision, and multimodal fusion. These tasks typically feature high computational intensity, high GPU memory requirements, and high dynamic real-time performance, and their efficient execution heavily relies on the parallel computing capabilities provided by graphics processing units (GPUs). As the core hardware carrier for AI computing tasks, the rationality of GPU resource scheduling directly affects task execution efficiency, resource utilization, and system stability. Furthermore, current AI tasks exhibit significant diversity: in terms of task type, they include model training and real-time inference; in terms of task parameters, they involve differences in computational precision and batch processing volume; and in terms of execution requirements, there are different timeliness requirements such as real-time response and offline batch processing. Against this backdrop, the precise matching of AI computing tasks with GPU resources has become crucial for improving system performance.

[0003] In related technologies, the amount of resources required to execute an AI task and the amount of idle resources on the GPU device are directly obtained. The AI ​​task is then allocated to a GPU device whose required resources are less than the amount of idle resources and scheduled for execution. However, during the allocation and scheduling process, multiple pieces of information, such as the task category of the AI ​​task, also affect the operation of the GPU device and the normal scheduling of the AI ​​task. Therefore, scheduling and allocating based solely on the amount of resources may lead to unreasonable allocation of AI tasks due to overly simplistic considerations. It is evident that the rationality of scheduling AI tasks in related technologies is poor. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a GPU resource scheduling method, apparatus, device, and medium.

[0005] Firstly, this application provides a GPU resource scheduling method, which adopts the following technical solution: A GPU resource scheduling method, comprising: In response to receiving an AI task, the device information of multiple AI tasks and multiple GPU devices is obtained, including the amount of resources that can be carried and the amount of resources to be released. The pre-trained task recognition model is used to analyze each AI task to obtain the required resources for the AI ​​task. The required resource quantities and the device information of each GPU device are matched to determine a number of target GPU devices from the multiple GPU devices. Obtain the current task content of each target GPU device, and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

[0006] In one embodiment, the training method of the task recognition model includes: Acquire multiple sample AI tasks, identify each sample AI task, and extract multiple task features from each sample AI task; Based on the softmax function, the task features of each task are predicted to obtain the task prediction result of the sample AI task. The task prediction result includes multiple task types and the probability distribution value of each task type. Obtain the actual task label of the sample AI task, and determine the adjusted task weight of the sample AI task based on the actual task label and the task prediction result; The task recognition model is adjusted according to the task weights to obtain the adjusted model. The adjusted task recognition model is trained iteratively a preset number of times, and the task recognition model after iterative training is determined as the completed task recognition model.

[0007] In one embodiment, determining the adjusted task weight of the sample AI task based on the actual task label and the prediction result includes: The sample AI task is determined to be a simple task or a complex task based on the actual task label. When the sample AI task is the complex task, compare the probability distribution values ​​of each task type in the sample AI task, and select the task type corresponding to the maximum probability distribution value as the predicted task type; The adjustment weights of the sample AI task are determined based on the maximum probability distribution value, the weight adjustment formula, and the preset adjustment coefficient.

[0008] In one embodiment, matching the demand information and the device information of each GPU device to determine the target GPU device from the plurality of GPU devices includes: Based on the required resource quantity and the resource capacity of each GPU device, several initial GPU devices are determined for each AI task. Obtain the execution date of each AI task, and determine the waiting time of each AI task based on the execution date; The target GPU device is determined based on the waiting time of each AI task and the amount of resources to be released from each initial GPU device.

[0009] In one embodiment, determining the execution strategy based on each AI task and the current task content of each target GPU device includes: Based on the task content of each AI task and the current task content of the target GPU device, the task similarity between each AI task and the target GPU device is determined; Task information is determined based on the task similarity and task similarity threshold. The task information includes several associated AI tasks and the execution order of each associated AI task. The task similarity of the associated AI tasks is greater than a preset similarity threshold. Based on the execution order of each associated AI task, the associated AI task in the first execution order is determined as the target AI task; Based on the task content of the target AI task and the task content of non-target AI tasks, determine a number of first AI tasks of the target AI task from the non-target AI tasks, and the degree of dependency between the target AI task and each of the first AI tasks. The non-target AI tasks are associated AI tasks that are not executed in the first order, and the first AI tasks are non-target AI tasks that have a dependency relationship with the target AI task. Based on the degree of dependency between the target AI task and each of the first AI tasks, a first execution order of each of the first AI tasks is determined; The first execution order of the target AI task and the first execution order of the first AI task are determined as the execution strategy.

[0010] In one embodiment, before determining the first execution order of the target AI task and the first execution order of each of the first AI tasks as the execution strategy, the method further includes: For the second AI task, the task execution time of each second AI task is determined according to the correspondence between the required resources and the task time of each second AI task. The second AI task is the non-target AI task that has no dependency relationship with the target AI task. The time priority of each second AI task is determined based on the task execution time and the execution date of each second AI task; Obtain the runtime environment information of each of the second AI tasks, and determine the runtime priority of each of the second AI tasks based on the runtime environment information; Based on the time priority and the running priority, determine the second execution order of each of the second AI tasks; Accordingly, determining the first execution order of the target AI task and the first execution order of each first AI task as the execution strategy includes: The execution strategy is determined by the first execution order of the target AI task, the first execution order of each first AI task, and the second execution order of each second AI task.

[0011] In one embodiment, if the task similarity between the AI ​​task and the target GPU device is no greater than a similarity threshold, the scheduling method further includes: Obtain the task logic corresponding to each of multiple unrelated AI tasks, and analyze each task logic to obtain the decomposition information of each task logic; If the splitting information indicates that the tasks can be split, then each of the unrelated AI tasks is split to obtain several sub-AI tasks; Obtain the resource requirements of each sub-AI task, and match the resource requirements of each sub-AI task with the resource capacity of each GPU device to determine the target GPU device for each sub-AI task.

[0012] Secondly, this application provides a GPU resource scheduling device, which adopts the following technical solution: A GPU resource scheduling device, comprising: The acquisition module is used to acquire device information of multiple AI tasks and multiple GPU devices in response to receiving an AI task. The device information includes the amount of resources that can be carried and the amount of resources to be released. The analysis module is used to analyze each of the AI ​​tasks using a pre-trained task recognition model to obtain the resource requirements of the AI ​​tasks. A matching module is used to match the requirement information and the device information of each GPU device to determine a number of target GPU devices from the multiple GPU devices; The determination module is used to obtain the current task content of each target GPU device and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

[0013] Thirdly, this application provides an electronic device that adopts the following technical solution: At least one processor; Memory; At least one application, stored in the memory, when executed by the at least one processor, causes the at least one processor to perform: executing the GPU resource scheduling method as described in any of the first aspects.

[0014] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform a GPU resource scheduling method as described in any of the first aspects.

[0015] In summary, this application includes the following beneficial technical effects: Upon receiving an AI task, this invention acquires device information for multiple AI tasks and multiple GPU devices, and uses a pre-trained task recognition model to analyze the AI ​​task to obtain the accurate resource requirements of the AI ​​task. Based on the accurate resource requirements, it matches the AI ​​task with each GPU device to find the correct target GPU device. Using the appropriate target GPU device to schedule and execute the AI ​​task can effectively improve the accuracy of AI task execution. Furthermore, it acquires the current task content of the target GPU device and determines the task execution strategy based on the task content to execute the AI ​​task in a targeted manner. Compared to related technologies that only match AI tasks, this application rearranges the AI ​​execution order after accurately matching AI tasks and GPU devices, effectively improving the accuracy of AI task scheduling. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a GPU resource scheduling method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a GPU resource scheduling device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The following is in conjunction with the appendix Figure 1 To be continued Figure 3 This application will be described in further detail.

[0018] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0021] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0022] With the rapid development of AI technology, the application of AI computing technology has become more widespread. AI computing can effectively simplify calculation steps and improve computational efficiency, and hardware resources play a crucial role in the AI ​​computing process. Currently, GPUs are the mainstream hardware resource for AI computing. However, to ensure high computational efficiency during the execution of AI computing tasks, it is necessary to rationally allocate GPU resources. Existing technologies directly obtain the idle resources of the GPU device. When the idle resources of the GPU device are sufficient to execute an AI computing task, the AI ​​computing task is scheduled to the GPU device for computation. However, when scheduling multiple AI computing tasks simultaneously, scheduling based solely on the idle resources of the GPU device may result in some AI computing tasks not being scheduled in time, thus affecting the efficiency of the AI ​​computing tasks. It is evident that the rationality of the current technology in scheduling AI computing tasks is poor.

[0023] Based on this, this application provides a technical solution to obtain AI task and GPU device information, and to use a pre-trained task recognition model to analyze the AI ​​task to obtain accurate AI task requirement information. Then, the requirement information and GPU device information are matched to obtain accurate GPU devices, and the task execution strategy is determined based on the GPU device and the current task content of the GPU device.

[0024] This application provides a GPU resource scheduling method executed by an electronic device, which can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication. This application does not impose any limitations on this connection. Figure 1 As shown, the method includes steps S1, S2, S3, and S4, wherein: Step S1: In response to receiving an AI task, obtain device information for multiple AI tasks and multiple GPU devices. The device information includes the amount of resources that can be carried and the amount of resources to be released.

[0025] Specifically, in the embodiments of this application, different application scenarios correspond to different AI tasks. For example, when the application scenario is model training, the AI ​​task is the computational task of training the model. When the application scenario is computer vision, the AI ​​task can be the task of analyzing and understanding images or videos. The device performance of multiple GPU devices can be the same or different.

[0026] Electronic devices have pre-integrated monitoring programs that monitor the triggering behavior of AI tasks. Once a request is detected, the program executes the acquisition operation. This involves retrieving device information for each GPU from a device information database, using the GPU's device identifier for matching. The available resource capacity refers to the remaining resources of the GPU, while the resource to be released is the amount of resources the GPU can release within a set timeframe. It's understood that the device information for each GPU can be the same or different; configuring GPUs with different device information allows for more flexible resource scheduling.

[0027] Step S2: Analyze each AI task using a pre-trained task recognition model to obtain the resource requirements of each AI task.

[0028] Specifically, the resource requirement represents the amount of GPU resources needed to execute an AI task. In this embodiment, the pre-trained task recognition model is obtained through iterative training using multiple historical data and a self-learning model. The specific training process can be referred to in the following embodiment. The AI ​​task is input into the pre-trained task recognition model, and the task recognition model outputs the task type of the AI ​​task. It can be understood that identifying AI tasks through the task recognition model can effectively improve the efficiency and accuracy of task type determination, thereby saving resource scheduling time, improving efficiency, and achieving rapid resource scheduling. The task type of the AI ​​task is matched with the correspondence between task type and resource requirement to obtain the resource requirement of the AI ​​task.

[0029] Step S3: Match the required resource quantities with the device information of each GPU device to determine several target GPU devices from multiple GPU devices.

[0030] Specifically, there may be one or more target GPU devices, and this application embodiment does not limit the number of devices. The specific process of matching the required resource quantity with the device information of the GPU devices to determine the target GPU device from multiple GPU devices can be referred to the following embodiment.

[0031] It is understandable that matching the target GPU device based on the resource requirements of the AI ​​task can effectively meet the scheduling needs of the AI ​​task. On this basis, when multiple AI tasks need to frequently interact with each other, communication delays will occur, and the difficulty of maintaining the GPU device will also increase. Therefore, it can be seen that assigning AI tasks to a GPU device for execution can effectively reduce the time loss during data transmission, greatly reduce the impact on the next AI task to be executed, and effectively improve the rationality of AI task allocation.

[0032] Step S4: Obtain the current task content of each target GPU device, and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

[0033] Specifically, the current task content can be task data of the currently executing task, or execution logic (such as model calculation, rendering instructions, etc.). The current task content of the GPU device can be obtained using API commands.

[0034] Specifically, the task execution strategy is determined based on the current task content of each AI task and each target GPU device, including: Based on the task content of each AI task and the current task content of the target GPU device, determine the task similarity between each AI task and the target GPU device; Task information is determined based on task similarity and task similarity threshold. The task information includes several related AI tasks and the execution order of each related AI task. The task similarity of the related AI tasks is greater than the preset similarity threshold. Based on the execution order of each associated AI task, the associated AI task with the first execution order is determined as the target AI task; Based on the task content of the target AI task and the task content of non-target AI tasks, determine several first AI tasks of the target AI task from the non-target AI tasks, as well as the degree of dependency between the target AI task and each first AI task. Among them, non-target AI tasks are related AI tasks that are not executed in the first order, and first AI tasks are non-target AI tasks that have a dependency relationship with target AI tasks; The first execution order of each first AI task is determined based on the degree of dependency between the target AI task and each first AI task. The execution order of the target AI task and the execution order of the first AI task are determined as the execution strategy.

[0035] Specifically, in one feasible approach, the process of determining the task similarity between each AI task and the target GPU device includes: identifying the task content of the AI ​​task and the task content of the target GPU device, and extracting AI task data and target GPU task data from the AI ​​task content; determining the similarity between the task data, and defining the data similarity as task similarity. Here, the task data is used to describe the key features of the task. For example, when the AI ​​task is the recognition of a smart camera, the task data can be resource data (memory requirement ≥6GB, must support CUDA 11.4+TensorRT acceleration). In another feasible implementation, the task similarity between each AI task and the target GPU device can be determined based on the execution logic. Specifically, this includes: obtaining a first execution logic and a second execution logic for the task content of the AI ​​task and the target GPU device; extracting a first key feature from the first execution logic and a second key feature from the second execution logic, where the key feature can be an operator sequence, data flow direction, or control structure, etc.; preprocessing the key features, such as converting the operator sequence into a vector, and performing similarity calculations, where the similarity of the operator sequence can be determined using the Levenshtein distance algorithm, etc. This application does not limit the specific determination process; existing technologies can be used. Finally, the similarities corresponding to all key features are summed to obtain the task similarity between each AI task and the target GPU device.

[0036] Next, the task similarity is compared with a task similarity threshold. AI tasks with a similarity greater than the threshold are identified as associated AI tasks, while those with a similarity less than or equal to the threshold are identified as unrelated tasks. The execution order of the associated AI tasks is determined in descending order of task similarity, meaning that AI tasks with higher similarity are executed earlier. The associated AI task corresponding to the first execution order is identified as the target AI task, and all associated AI tasks other than the target AI task are identified as non-target AI tasks. The specific process for determining the dependencies between several first AI tasks of the target AI task, the target AI task, and each first AI task includes: obtaining first task labels for the target AI task and second task labels for non-target AI tasks, wherein the first and second task labels are used to describe the task content of the AI ​​task and are pre-set by relevant technical personnel; determining the dependencies between the target AI task and each non-target AI task based on the first and second task labels; when the target AI task and non-target AI tasks have a sequential dependency relationship, it corresponds to a first degree of dependency; when the target AI task and non-target AI tasks have a parallel dependency relationship, it corresponds to a second degree of dependency; when the target AI task and non-target AI tasks have no dependency relationship, it corresponds to a third degree of dependency; wherein the first degree of dependency > the second degree of dependency > the third degree of dependency, and the value corresponding to the third degree of dependency is 0. For the target AI task, the execution order of non-target tasks with the first degree of dependency is determined as the second execution order; then, the dependency degree of the non-target AI tasks in the second execution order is determined with other non-target AI tasks, and the execution order of each non-target AI task is determined in turn. Thus, non-target AI tasks that have a dependency relationship with the target AI task and non-target AI tasks that have a dependency relationship with each other can be obtained. Non-target AI tasks that have a dependency relationship with the target AI task and non-target AI tasks that have a dependency relationship with other AI tasks are determined as first AI tasks, and non-target AI tasks that do not have a dependency relationship with any AI task are determined as second AI tasks. The task similarity threshold is preset, and the specific value of the task similarity threshold is not limited in this embodiment. Further, AI tasks with a serial dependency relationship are those that require the execution of the previous AI task to start the next AI task, and the input data of the next AI task is the output result of the previous AI task; AI tasks with a parallel dependency relationship refer to AI tasks that can be started and executed simultaneously.

[0037] Furthermore, for AI tasks with parallel dependencies, the execution order is determined according to the sequence of the AI ​​tasks. For example, in AI1, AI2, AI3, AI4, AI5, and AI6, AI1 is the target AI task, and AI2 has a serial dependency on AI1; therefore, the execution order of AI2 is the second execution order. Next, the dependencies between AI2 and AI3, AI4, AI5, and AI6 are determined, and a third execution order is determined based on these dependencies, until the execution order for each task is determined. If AI6 has a parallel dependency on AI1, and AI5 has a parallel dependency on AI2, then the tasks are scheduled in the order of AI1, AI2, AI3, AI4, AI6, and AI5. It is understandable that determining the execution order of AI tasks according to their dependencies and scheduling them sequentially not only ensures the correctness of the task logic and avoids execution errors but also reduces the overall latency. When the content of an AI task is the same as the content of the task currently being executed by the GPU, the context switching overhead of the GPU can be effectively reduced, and the utilization of video memory can also be improved. Therefore, AI tasks with high task similarity are prioritized for execution and identified as target AI tasks. Then, non-target AI tasks with dependencies are identified based on the content of the target AI task and the first AI task. Prioritizing the execution of non-target AI tasks with dependencies during the scheduling and execution of AI tasks can also reduce the context switching overhead of the GPU. Therefore, the AI ​​tasks are executed sequentially according to the first execution order of the target AI task and each first AI task, which effectively improves the rationality of AI task execution.

[0038] Furthermore, for the second AI task, the execution order of each second AI task is determined in ascending order according to the resource requirements of each second AI task. The first execution order of the target AI task, the first execution order of each first AI task, and the execution order of each second AI task are determined as the execution strategy and scheduled for execution.

[0039] Based on the above embodiments, in response to receiving an AI task, device information of multiple AI tasks and multiple GPU devices is obtained, and the AI ​​task is analyzed using a pre-trained task recognition model to obtain the accurate resource requirements of the AI ​​task. The accurate resource requirements are used as a reference basis to match with each GPU device to find the correct target GPU device. Scheduling and executing the AI ​​task on the appropriate target GPU device can effectively improve the accuracy of AI task execution. Then, the current task content of the target GPU device is obtained, and the task execution strategy is determined according to the task content so as to execute the AI ​​task in a targeted manner. Compared with related technologies that only match AI tasks, this application rearranges the AI ​​execution order on the basis of accurately matching AI tasks and GPU devices, which effectively improves the accuracy of AI task scheduling.

[0040] One possible implementation of this application embodiment, the training method of the task recognition model, includes: Acquire multiple sample AI tasks, identify each sample AI task, and extract multiple task features from each sample AI task; Based on the softmax function, the features of each task are predicted to obtain the task prediction results of the sample AI task. The task prediction results include multiple task types and the probability distribution values ​​of each task type. Obtain the actual task labels of the sample AI tasks, and adjust the learning task weights of each task type in the task recognition model based on the actual task labels and task prediction results. The task recognition model with adjusted task weights is trained iteratively a preset number of times, and the task recognition model after iterative training is determined as the completed task recognition model.

[0041] Specifically, the sample AI task is an AI task that has been scheduled and used for model training. Sample AI tasks can be obtained from a historical task information database. The shared feature layer in the task recognition model extracts task features from the sample AI tasks. This application does not limit the specific process of using the softmax function to predict the task features of the sample AI tasks; users can set this themselves. Using the softmax function to predict each sample AI task yields several task types and corresponding probability distribution values ​​for each task type. The probability distribution value describes the probability that the sample AI task belongs to this task type; a higher probability distribution value indicates a higher task type. For example, if the predicted result of the sample AI task is task type 1 and task type 2, with corresponding probability distribution values ​​of 50% and 80%, it indicates that the probability of the sample AI task being task type 2 is higher. The actual task label is pre-defined by the technician and is used to describe the actual task type of the sample AI task. The specific process of adjusting the learning task weights based on the actual task label and task prediction results can be found in the following embodiment. It is understandable that adjusting the learning task weights allows the model to focus on recognizing the specific task type, thereby improving the accuracy of task type recognition. The specific process of adjusting the task weights to obtain the adjusted task recognition model includes: differentiating the maximum probability distribution value based on a preset error function, then differentiating the adjustment weights using the maximum probability distribution value to obtain partial derivatives; determining the target adjustment range corresponding to the partial derivatives based on the correspondence between the partial derivatives, the partial derivative values, and the adjustment range; and finally, adjusting according to the target adjustment range. It is understood that the input layer and hidden layer are connected by neurons, and each neuron corresponds to a weight value. The original weight values ​​are adjusted according to the target adjustment range to obtain the adjusted task recognition model. The preset number of iterations is predetermined by the technician; this embodiment does not limit the preset number of iterations. By performing a preset number of iterations, the output effect of the task recognition model can be obtained to achieve the preset effect, and the task recognition model after reaching the preset number of iterations is determined as the successfully trained task recognition model.

[0042] Based on the above embodiments, multiple sample AI tasks are obtained and identified to extract multiple task features from them. Then, the task features of the sample AI tasks are predicted using the softmax function to obtain the prediction results. The actual task labels of the sample AI tasks are obtained, and the task weights are adjusted according to the actual task labels and the task prediction results. When there is a large deviation between the task prediction results and the actual task labels, it indicates that the accuracy of the model's prediction results is poor. At this time, it is necessary to adjust the model for this task to focus on identifying it until the task recognition model can accurately identify the AI ​​tasks. The task recognition model after iterative training is then determined as the trained task recognition model, which effectively improves the accuracy of the task recognition model.

[0043] One possible implementation of this application embodiment involves determining the adjusted task weights of the sample AI task based on the actual task label and prediction results, including: The sample AI task is determined to be either a simple task or a complex task based on the actual task label. When the sample AI task is a complex task, compare the probability distribution values ​​of each task type in the sample AI task, and select the task type corresponding to the maximum probability distribution value as the predicted task type; The adjustment weights for the sample AI task are determined based on the maximum probability distribution value, the weight adjustment formula, and the preset adjustment coefficient.

[0044] Specifically, for the sample AI task, the task is determined to be either a simple or complex task based on the correspondence between the sample AI task, the actual task label, and the task. When the AI ​​task is a complex task, the probability distribution values ​​corresponding to each task type are compared, and the maximum probability distribution value is determined from multiple probability distribution values. The specific process of determining the adjustment weight of the sample AI task based on the maximum probability distribution value, the weight adjustment formula, and the preset adjustment coefficient includes: Adjustment weight = (maximum probability distribution value - probability distribution value threshold) * preset adjustment coefficient, where the preset adjustment coefficient is 0.14 and the probability distribution value threshold is 0.5 in this embodiment. When the sample AI task is a simple task, the task recognition model is trained a preset number of times directly.

[0045] Based on the above embodiments, the task category corresponding to the first maximum confidence level is selected as the predicted task category. The difficulty of identifying the sample AI task is distinguished according to the confidence level of the predicted task category and the preset confidence threshold. When the predicted type of the sample AI task is a difficult task, the second maximum confidence level corresponding to the difficult task is obtained. The weight of the sample AI task is adjusted according to the second maximum confidence level, the weight adjustment formula and the preset adjustment coefficient, so as to obtain the accurate task weight of the task category corresponding to the sample AI task, and thus more accurately identify the task category.

[0046] One possible implementation of this application embodiment involves matching the demand information and the device information of each GPU device to determine the target GPU device from multiple GPU devices, including: Based on the required resources and the available resources of each GPU device, several initial GPU devices are determined for each AI task. Obtain the execution date of each AI task and determine the waiting time for each AI task based on the execution date; The target GPU device is determined based on the waiting time of each AI task and the amount of resources to be released from each initial GPU device.

[0047] Specifically, the required resources represent the amount of resources needed to schedule AI tasks, while the available resources represent the current amount of idle resources on the GPU device. The required resources and available resources are compared one by one, and GPU devices with available resources greater than required resources are identified as the initial GPU devices. The execution date represents the latest execution date of the AI ​​task; the current date is obtained, and the time difference between the current date and the execution time is determined as the waiting time. The amount of resources to be released is the amount of resources released by the AI ​​tasks that the initial GPU device can complete within the waiting time. When the initial GPU device completes more AI tasks, it can release more resources to be released, thus allowing it to execute more AI tasks; therefore, the amount of resources to be released needs to be considered. The required resources and the amount of resources to be released for each AI task are compared, and AI tasks with the amount of resources to be released greater than required resources are identified as the target GPU devices, thus obtaining a more accurate target GPU device.

[0048] Based on the above embodiments, an initial screening of GPU devices is performed according to the required resources and the amount of resources that each GPU device can support, so as to ensure that the initial GPU devices can meet the resource requirements of the AI ​​tasks; then the execution time of each AI task is obtained, and a second screening of GPU devices is performed according to the waiting time and the amount of resources to be released of each initial GPU device, thereby effectively improving the accuracy of GPU device matching.

[0049] One possible implementation of this application embodiment, before determining the first execution order of the target AI task and the first execution order of each first AI task as the execution strategy, further includes: For the second AI task, the execution time of each second AI task is determined based on the correspondence between the required resources, the required resources and the task time. The second AI task is a non-target AI task that has no dependency on the target AI task. The time priority of each second AI task is determined based on the task execution time and execution date of each second AI task; Obtain the runtime environment information of each second AI task, and determine the runtime priority of each second AI task based on the runtime environment information; The second execution order of each second AI task is determined based on time priority and execution priority; Accordingly, the first execution order of the target AI task and the first execution order of each first AI task are determined as the execution strategy, including: The execution strategy is determined by the first execution order of the target AI task, the first execution order of each first AI task, and the second execution order of each second AI task.

[0050] Specifically, the correspondence between resource requirements and task duration is pre-defined by technical personnel. It is understood that as resource requirements increase, task duration also increases, thus determining the execution time of each second AI task. The specific process for determining the time priority of each second AI task includes: determining a first time priority corresponding to the task execution time based on the correspondence between task execution time, execution duration, and time priority; determining a second time priority corresponding to the execution date based on the correspondence between the execution date, execution date, and time priority; determining a comprehensive time priority based on the first and second time priorities; and setting the comprehensive time priority as the time priority for each second AI task. In this embodiment, the longer the task execution time, the higher the first time priority, meaning the task is executed first; the later the execution date, the lower the second time priority. The runtime environment information represents whether a task needs to run in a specific software framework or toolkit (e.g., TensorFlow, PyTorch) or not. It is understood that running in a specific software framework or toolkit requires more resources, thus necessitating the acquisition of runtime environment information. The operating environment can be determined based on the correspondence between operating priority and operating environment. This correspondence is set by technical personnel, where the operating priority of AI tasks in a specific operating environment is higher than that of AI tasks in a non-specific operating environment. Further, a second execution order for each second AI task can be obtained by weighted accumulation based on time priority and operating priority. This application embodiment does not limit the specific process of the second execution order. In this application embodiment, the second execution order corresponding to all second AI tasks is after the first execution order of the first AI tasks, and then the AI ​​tasks are scheduled and executed sequentially according to the execution order of the target AI task and non-target AI tasks. It is understood that during resource scheduling, the resource requirements of AI tasks affect task execution time; the higher the task execution time, the greater the impact on the execution of other AI tasks. Therefore, it is necessary to determine the task execution time of AI tasks.

[0051] Based on the above embodiments, before determining the execution strategy, the priority of the second AI task is determined from the time dimension by referring to the resource requirements and task consumption of the second AI task. When the AI ​​task needs to run under a specific software framework, the more resources the second AI task needs to consume, the longer the waiting time of other second AI tasks following the second AI task will be. This may lead to task congestion. Therefore, it is necessary to determine the running priority according to the running environment of the second AI task. Then, by comprehensively analyzing the time priority and running priority, a more accurate second execution order of the second AI task is obtained. The execution order of each AI task is determined from different dimensions for different situations of AI tasks and executed in sequence. Compared with the method of using a uniform evaluation for all AI tasks, this application effectively improves the rationality of the execution scheduling of each AI task.

[0052] In one possible implementation of this application embodiment, if the task similarity between the AI ​​task and the target GPU device is no greater than a similarity threshold, the scheduling method further includes: Obtain the task logic of multiple unrelated AI tasks, and analyze the task logic to obtain the decomposition information corresponding to the task logic; If the information can be split, then split each unrelated AI task to obtain several sub-AI tasks; Obtain the resource requirements of each sub-AI task, and match the resource requirements of each sub-AI task with the resource capacity of each GPU device to determine the target GPU device for each sub-AI task.

[0053] Specifically, the task logic of unrelated AI tasks can be located from the requirements document or code comments. When an unrelated AI task has multiple task logics, the corresponding splitting information indicates that it can be split; when an unrelated AI task has only one task logic, the corresponding splitting information indicates that it cannot be split. Further, when the splitting information indicates that it can be split, the unrelated AI task is split into multiple sub-AI tasks. Then, based on the resource requirements of each sub-AI task, the resource requirements of the sub-AI task are matched with the idle resources of each GPU device, and the GPU device with idle resources greater than the resource requirements is identified as the target GPU device for the sub-AI task. It is understandable that splitting the task into multiple sub-tasks and assigning them to different GPU devices not only avoids GPU device overload but also reduces the waiting time of unrelated AI tasks, preventing excessive waiting. Further, if the splitting information for the unrelated AI task logic indicates that it cannot be split, the unrelated AI task is added to a queuing pool until any GPU device has idle resources greater than the resource requirements of the unrelated AI task, and that GPU device is identified as the target GPU device. Based on the above embodiments, for unrelated AI tasks with low task similarity, it is determined whether the unrelated AI tasks can be split. If they can be split, the unrelated AI tasks are split into several sub-tasks. By splitting, the unrelated AI tasks are reasonably allocated. This not only reduces the waiting time of unrelated AI tasks, but also avoids GPU overload, effectively improving the rationality of AI task allocation.

[0054] The above embodiments describe a GPU resource scheduling method from the perspective of process flow. The following embodiments describe a GPU resource scheduling device from the perspective of virtual modules or virtual units. For details, please refer to the following embodiments.

[0055] This application provides a GPU resource scheduling device, such as... Figure 2 As shown, the GPU resource scheduling device may specifically include: The acquisition module 201 is used to acquire device information of multiple AI tasks and multiple GPU devices in response to receiving an AI task. The device information includes the amount of resources that can be carried and the amount of resources to be released. Analysis module 202 is used to analyze each AI task using a pre-trained task recognition model to obtain the requirement information of the AI ​​task, including the required amount of resources. The matching module 203 is used to match the demand information and the device information of each GPU device to determine several target GPU devices from multiple GPU devices. The determination module 204 is used to obtain the current task content of each target GPU device and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

[0056] Based on the above embodiments, in response to receiving an AI task, device information of multiple AI tasks and multiple GPU devices is obtained, and the AI ​​task is analyzed using a pre-trained task recognition model to obtain the accurate resource requirements of the AI ​​task. The accurate resource requirements are used as a reference basis to match with each GPU device to find the correct target GPU device. Scheduling and executing the AI ​​task on the appropriate target GPU device can effectively improve the accuracy of AI task execution. Then, the current task content of the target GPU device is obtained, and the task execution strategy is determined according to the task content so as to execute the AI ​​task in a targeted manner. Compared with related technologies that only match AI tasks, this application rearranges the AI ​​execution order on the basis of accurately matching AI tasks and GPU devices, which effectively improves the accuracy of AI task scheduling.

[0057] One possible implementation of this application embodiment, the GPU resource scheduling device, further includes: The model training module is used for: Acquire multiple sample AI tasks, identify each sample AI task, and extract multiple task features from each sample AI task; Based on the softmax function, the features of each task are predicted to obtain the task prediction results of the sample AI task. The task prediction results include multiple task types and the probability distribution values ​​of each task type. Obtain the actual task labels of the sample AI tasks, and determine the adjusted task weights of the sample AI tasks based on the actual task labels and task prediction results. The task recognition model is adjusted by adjusting the task weights to obtain the adjusted model. The adjusted task recognition model is trained iteratively a predetermined number of times, and the task recognition model after iterative training is determined as the completed task recognition model.

[0058] In one possible implementation of this application embodiment, the model training module, when performing the task weight adjustment based on the actual task label and prediction results to determine the sample AI task, is used for: The sample AI task is determined to be either a simple task or a complex task based on the actual task label. When the sample AI task is a complex task, compare the probability distribution values ​​of each task type in the sample AI task, and select the task type corresponding to the maximum probability distribution value as the predicted task type; The adjustment weights for the sample AI task are determined based on the maximum probability distribution value, the weight adjustment formula, and the preset adjustment coefficient.

[0059] In one possible implementation of this application embodiment, when the matching module 203 performs the matching of each demand information and the device information of each GPU device to determine the target GPU device from multiple GPU devices, it is used to: Based on the required resources and the available resources of each GPU device, several initial GPU devices are determined for each AI task. Obtain the execution date of each AI task and determine the waiting time for each AI task based on the execution date; The target GPU device is determined based on the waiting time of each AI task and the amount of resources to be released from each initial GPU device.

[0060] In one possible implementation of this application embodiment, when determining the execution strategy based on the current task content of each AI task and each target GPU device, the determining module 204 is used to: Based on the task content of each AI task and the current task content of the target GPU device, determine the task similarity between each AI task and the target GPU device; Task information is determined based on task similarity and task similarity threshold. The task information includes several related AI tasks and the execution order of each related AI task. The task similarity of the related AI tasks is greater than the preset similarity threshold. Based on the execution order of each associated AI task, the associated AI task with the first execution order is determined as the target AI task; Based on the task content of the target AI task and the task content of the non-target AI task, determine several first AI tasks of the target AI task from the non-target AI tasks, the degree of dependency between the target AI task and each first AI task, the non-target AI task is the associated AI task that is not executed in the first order, and the first AI task is the non-target AI task that has a dependency relationship with the target AI task. The first execution order of each first AI task is determined based on the degree of dependency between the target AI task and each first AI task. The first execution order of the target AI task and the first execution order of the first AI task are determined as the execution strategy.

[0061] One possible implementation of this application embodiment, the GPU resource scheduling device, further includes: The correction module is used for: For the second AI task, the execution time of each second AI task is determined based on the correspondence between the required resources, the required resources and the task time. The second AI task is a non-target AI task that has no dependency on the target AI task. The time priority of each second AI task is determined based on the task execution time and execution date of each second AI task; Obtain the runtime environment information of each second AI task, and determine the runtime priority of each second AI task based on the runtime environment information; The second execution order of each second AI task is determined based on time priority and execution priority; Accordingly, the first execution order of the target AI task and the first execution order of each first AI task are determined as the execution strategy, including: The execution strategy is determined by the first execution order of the target AI task, the first execution order of each first AI task, and the second execution order of each second AI task.

[0062] One possible implementation of this application embodiment, the GPU resource scheduling device, further includes: Split modules, used for: Obtain the task logic corresponding to each of multiple unrelated AI tasks, and analyze the task logic to obtain the decomposition information of each task logic; If the information can be split, then split each unrelated AI task to obtain several sub-AI tasks; Obtain the resource requirements of each sub-AI task, and match the resource requirements of each sub-AI task with the resource capacity of each GPU device to determine the target GPU device for each sub-AI task.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the GPU resource scheduling device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0064] This application provides an electronic device, such as... Figure 3 As shown, Figure 3 The illustrated electronic device includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device does not constitute a limitation on the embodiments of this application.

[0065] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0066] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0067] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0068] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0069] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0070] This application provides a computer-readable storage medium storing a computer program. When the program is run on a computer, it enables the computer to execute the corresponding content in the aforementioned method embodiments. Compared to related technologies, in response to receiving an AI task, this application acquires device information for multiple AI tasks and multiple GPU devices, and uses a pre-trained task recognition model to analyze the AI ​​task to obtain the accurate resource requirements of the AI ​​task. Using the accurate resource requirements as a reference, it matches the AI ​​task with each GPU device to find the correct target GPU device. Scheduling and executing the AI ​​task on the appropriate target GPU device can effectively improve the accuracy of AI task execution. Furthermore, the application acquires the current task content of the target GPU device and determines the task execution strategy based on the task content to execute the AI ​​task in a targeted manner. Compared to related technologies that only match AI tasks, this application rearranges the AI ​​execution order based on accurate matching of AI tasks and GPU devices, effectively improving the accuracy of AI task scheduling.

[0071] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0072] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A GPU resource scheduling method, characterized in that, include: In response to receiving an AI task, the device information of multiple AI tasks and multiple GPU devices is obtained, including the amount of resources that can be carried and the amount of resources to be released. The pre-trained task recognition model is used to analyze each AI task to obtain the required resources for the AI ​​task. The required resource quantities and the device information of each GPU device are matched to determine a number of target GPU devices from the multiple GPU devices. Obtain the current task content of each target GPU device, and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

2. The GPU resource scheduling method as described in claim 1, characterized in that, The training methods for the task recognition model include: Acquire multiple sample AI tasks, identify each sample AI task, and extract multiple task features from each sample AI task; Based on the softmax function, the task features of each task are predicted to obtain the task prediction result of the sample AI task. The task prediction result includes multiple task types and the probability distribution value of each task type. Obtain the actual task label of the sample AI task, and determine the adjusted task weight of the sample AI task based on the actual task label and the task prediction result; The task recognition model is adjusted according to the task weights to obtain the adjusted model. The adjusted task recognition model is trained iteratively a preset number of times, and the task recognition model after iterative training is determined as the completed task recognition model.

3. The GPU resource scheduling method as described in claim 2, characterized in that, The step of determining the adjusted task weight of the sample AI task based on the actual task label and the prediction result includes: The sample AI task is determined to be a simple task or a complex task based on the actual task label. When the sample AI task is the complex task, compare the probability distribution values ​​of each task type in the sample AI task, and select the task type corresponding to the maximum probability distribution value as the predicted task type; The adjustment weights of the sample AI task are determined based on the maximum probability distribution value, the weight adjustment formula, and the preset adjustment coefficient.

4. The GPU resource scheduling method as described in claim 1, characterized in that, The step of matching the demand information and the device information of each GPU device to determine the target GPU device from the plurality of GPU devices includes: Based on the required resource quantity and the resource capacity of each GPU device, several initial GPU devices are determined for each AI task. Obtain the execution date of each AI task, and determine the waiting time of each AI task based on the execution date; The target GPU device is determined based on the waiting time of each AI task and the amount of resources to be released from each initial GPU device.

5. The GPU resource scheduling method as described in claim 1, characterized in that, The step of determining the execution strategy based on the current task content of each AI task and each target GPU device includes: Based on the task content of each AI task and the current task content of the target GPU device, the task similarity between each AI task and the target GPU device is determined; Task information is determined based on the task similarity and task similarity threshold. The task information includes several associated AI tasks and the execution order of each associated AI task. The task similarity of the associated AI tasks is greater than a preset similarity threshold. Based on the execution order of each associated AI task, the associated AI task in the first execution order is determined as the target AI task; Based on the task content of the target AI task and the task content of non-target AI tasks, determine a number of first AI tasks of the target AI task from the non-target AI tasks, and the degree of dependency between the target AI task and each of the first AI tasks. The non-target AI tasks are associated AI tasks that are not executed in the first order, and the first AI tasks are non-target AI tasks that have a dependency relationship with the target AI task. Based on the degree of dependency between the target AI task and each of the first AI tasks, a first execution order of each of the first AI tasks is determined; The first execution order of the target AI task and the first execution order of the first AI task are determined as the execution strategy.

6. The GPU resource scheduling method as described in claim 5, characterized in that, Before determining the first execution order of the target AI task and the first execution order of each of the first AI tasks as the execution strategy, the method further includes: For the second AI task, the task execution time of each second AI task is determined according to the correspondence between the required resources and the task time of each second AI task. The second AI task is the non-target AI task that has no dependency relationship with the target AI task. The time priority of each second AI task is determined based on the task execution time and the execution date of each second AI task; Obtain the runtime environment information of each of the second AI tasks, and determine the runtime priority of each of the second AI tasks based on the runtime environment information; Based on the time priority and the running priority, determine the second execution order of each of the second AI tasks; Accordingly, determining the first execution order of the target AI task and the first execution order of each first AI task as the execution strategy includes: The execution strategy is determined by the first execution order of the target AI task, the first execution order of each first AI task, and the second execution order of each second AI task.

7. The GPU resource scheduling method as described in claim 5, characterized in that, If the task similarity between the AI ​​task and the target GPU device is no greater than a similarity threshold, the scheduling method further includes: Obtain the task logic corresponding to each of multiple unrelated AI tasks, and analyze each task logic to obtain the decomposition information of each task logic; If the splitting information indicates that the tasks can be split, then each of the unrelated AI tasks is split to obtain several sub-AI tasks; Obtain the resource requirements of each sub-AI task, and match the resource requirements of each sub-AI task with the resource capacity of each GPU device to determine the target GPU device for each sub-AI task.

8. A GPU resource scheduling device, characterized in that, include: The acquisition module is used to acquire device information of multiple AI tasks and multiple GPU devices in response to receiving an AI task. The device information includes the amount of resources that can be carried and the amount of resources to be released. The analysis module is used to analyze each of the AI ​​tasks using a pre-trained task recognition model to obtain the resource requirements of the AI ​​tasks. A matching module is used to match the requirement information and the device information of each GPU device to determine a number of target GPU devices from the multiple GPU devices; The determination module is used to obtain the current task content of each target GPU device and determine the task execution strategy based on each AI task and the current task content of each target GPU device.

9. An electronic device, characterized in that, include: At least one processor; Memory; At least one application is stored in the memory, and when the at least one application is executed by the at least one processor, the at least one processor performs the GPU resource scheduling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed in a computer, causes the computer to perform the GPU resource scheduling method according to any one of claims 1 to 7.

Images