A task scheduling method, device, apparatus and storage medium

Abstract

The application discloses a task scheduling method and device, equipment and a storage medium, relates to the technical field of resource management, and is applied to a Kubernetes cluster and comprises the following steps: determining a current cloud computing task to be scheduled; determining a candidate physical neural network processor according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to a physical neural network processor in the current cluster; determining a target physical neural network processor and a target scheduling node from the candidate physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request; and creating virtual resource information through the target scheduling node to execute the current cloud computing task. Therefore, the application schedules cloud computing tasks by means of virtualization technology and virtual resources corresponding to physical neural network processors, dynamically allocates virtual resources according to the running state of the cluster, and improves resource utilization and task processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource management, and particularly relates to a task scheduling method and device, equipment and a storage medium. BACKGROUND

[0002] Traditional Ascend NPU (Ascend Neural Processing Unit, a neural network processor for high-performance AI computing) resource management usually adopts a static allocation method, that is, fixed NPU resources are allocated to tasks according to a pre-defined configuration. This method may perform well in the case of small task size and stable resource demand, but in the large-scale AI batch task processing scenario, it has significant limitations. First, static allocation can easily lead to uneven resource utilization, and some Ascend NPUs may be overloaded due to high task load, while other NPUs are idle, causing resource waste. Secondly, with the dynamic changes of AI task demand, the static allocation method is difficult to adjust the resource allocation flexibly, and cannot meet the immediate demand of AI tasks for resources.

[0003] In order to overcome these limitations, the industry has begun to explore resource dynamic scheduling methods based on cloud computing. Kubernetes, as a popular container orchestration and management platform, provides powerful resource scheduling and automatic expansion capabilities, providing a new idea for the dynamic management of Ascend NPU resources. However, directly applying Kubernetes to the scheduling of Ascend NPU resources also faces many challenges. On the one hand, Ascend NPU resources have higher professionalism and complexity compared to traditional CPU (Central Processing Unit) and GPU (Graphic Processing Unit) resources, and require more refined management and scheduling strategies. On the other hand, large-scale AI batch tasks usually contain multiple parallel sub-tasks, and these sub-tasks may have complex dependency relationships and resource competition.

[0004] Therefore, how to efficiently schedule cloud computing tasks and improve resource utilization is a problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a task scheduling method, device, equipment and storage medium, which can schedule and execute cloud computing tasks by means of virtualization technology through virtual resource information corresponding to physical neural network processors, dynamically allocate virtual resources according to the running situation of the cluster, and improve resource utilization and processing efficiency of cloud computing tasks. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a task scheduling method applied to a Kubernetes cluster, comprising:

[0007] determining a current cloud computing task to be scheduled;

[0008] determining a candidate physical neural network processor corresponding to the current cloud computing task according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to a physical neural network processor in a current cluster;

[0009] determining a target physical neural network processor corresponding to the current cloud computing task and a target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request;

[0010] creating virtual resource information corresponding to the virtual resource request through the target scheduling node, so as to execute the current cloud computing task by using the virtual resource information.

[0011] Optionally, the determining of the current cloud computing task to be scheduled comprises:

[0012] performing priority sorting on subtasks in a cloud computing batch task obtained according to a number of virtual neural network processors and a video memory value in a virtual resource request corresponding to a cloud computing task, to obtain a priority queue corresponding to the cloud computing batch task;

[0013] performing iteration on the subtasks in the priority queue to determine a current subtask to be scheduled as the current cloud computing task.

[0014] Optionally, the determining of the candidate physical neural network processor corresponding to the current cloud computing task according to the matching relationship between the virtual resource request corresponding to the current cloud computing task and the physical resource information corresponding to the physical neural network processor in the current cluster comprises:

[0015] determining a matching virtual device type from the physical resource information corresponding to the physical neural network processor in the current cluster according to a number of virtual neural network processors and a video memory value requested by the current cloud computing task; the virtual device type is a device type corresponding to a virtual neural network processor obtained by virtualization division of the physical neural network processor;

[0016] determining the physical neural network processor corresponding to the virtual device type as the candidate physical neural network processor corresponding to the current cloud computing task.

[0017] Optionally, the determining, based on the preset processor scoring rule and in combination with the virtual resource request, the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors and a target scheduling node corresponding to the target physical neural network processor, comprises:

[0018] The preset processor scoring rule is used to score the candidate physical neural network processors to obtain corresponding processor scores.

[0019] The target physical neural network processor corresponding to the current cloud computing task is determined from the candidate physical neural network processors based on the virtual resource request, the processor scores and the number of virtual neural network processors corresponding to the candidate physical neural network processors, and a target scheduling node corresponding to the target physical neural network processor is determined.

[0020] Optionally, the determining, based on the virtual resource request, the processor scores and the number of virtual neural network processors corresponding to the candidate physical neural network processors, the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors, comprises:

[0021] The candidate physical neural network processors are sorted based on the principle that the processor scores are in descending order to obtain a corresponding first list.

[0022] The first list is traversed according to the number of virtual neural network processors corresponding to the virtual resource request to determine the target physical neural network processor corresponding to the current cloud computing task from the first list.

[0023] Optionally, the creating, by the target scheduling node, virtual resource information corresponding to the virtual resource request so as to execute the current cloud computing task by using the virtual resource information, comprises:

[0024] The target virtual neural network processor corresponding to the virtual resource request is created by the target scheduling node, and the target virtual neural network processor is mounted to the current cloud computing task.

[0025] The current cloud computing task is executed by using the target virtual neural network processor.

[0026] Optionally, before the determining the current cloud computing task to be scheduled, the method further comprises:

[0027] Obtaining a cloud computing batch task.

[0028] determine whether the name of the virtual neural network processor requested by the cloud computing batch task exists in the physical resource information, to obtain a first determination result; the virtual neural network processor is a processor obtained by virtualizing and dividing a physical neural network processor;

[0029] If the first determination result indicates that the name of the corresponding virtual neural network processor exists in the physical resource information, determine whether the number of virtual neural network processors requested by the cloud computing batch task is greater than a preset number threshold, to obtain a second determination result;

[0030] If the second determination result indicates that the number of virtual neural network processors requested by the cloud computing batch task is not greater than the preset number threshold, determine whether the value of the corresponding video memory of the virtual neural network processor requested by the cloud computing batch task is greater than a preset video memory threshold, to obtain a third determination result;

[0031] If the third determination result indicates that the value of the corresponding video memory of the virtual neural network processor requested by the cloud computing batch task is not greater than the preset video memory threshold, determine that the cloud computing batch task is a to-be-scheduled batch task, so as to determine a to-be-scheduled current cloud computing task from the to-be-scheduled batch task.

[0032] In a second aspect, the present application provides a task scheduling device applied to a Kubernetes cluster, comprising:

[0033] a task determination module configured to determine a to-be-scheduled current cloud computing task;

[0034] a first processor determination module configured to determine a to-be-selected physical neural network processor corresponding to the current cloud computing task according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to a physical neural network processor in the current cluster;

[0035] a second processor determination module configured to determine a target physical neural network processor corresponding to the current cloud computing task and a target scheduling node corresponding to the target physical neural network processor from the to-be-selected physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request;

[0036] a task execution module configured to create virtual resource information corresponding to the virtual resource request through the target scheduling node, so as to execute the current cloud computing task by using the virtual resource information.

[0037] In a third aspect, the present application provides an electronic device, comprising:

[0038] a memory configured to save a computer program;

[0039] A processor is configured to execute the computer program to implement the task scheduling method as described above.

[0040] In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program, which, when executed by a processor, implements the task scheduling method as described above.

[0041] As can be seen, in the present application, the Kubernetes cluster first determines a current cloud computing task to be scheduled; then determines a candidate physical neural network processor corresponding to the current cloud computing task according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to the physical neural network processor in the current cluster; determines a target physical neural network processor corresponding to the current cloud computing task and a target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request; then creates virtual resource information corresponding to the virtual resource request through the target scheduling node, so as to execute the current cloud computing task by using the virtual resource information. In this way, the present application can schedule and execute the cloud computing task by means of the virtualization technology and the virtual resource information corresponding to the physical neural network processor, so as to improve the resource utilization rate of the physical neural network processor; and in combination with the processor scoring rule, the virtual resource can be dynamically allocated according to the running condition of the cluster, so as to improve the processing efficiency of the cloud computing task. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0043] Figure 1 A task scheduling method flow chart disclosed by the present application;

[0044] Figure 2 A specific task scheduling method flow chart disclosed by the present application;

[0045] Figure 3 A task scheduling device structure schematic diagram disclosed by the present application;

[0046] Figure 4 A structure diagram of an electronic device disclosed by the present application. DETAILED DESCRIPTION

[0047] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0048] Referring to Figure 1 As shown in the drawings, the embodiments of the present application disclose a task scheduling method, applied to a Kubernetes cluster, comprising:

[0049] In step S11, a current cloud computing task to be scheduled is determined.

[0050] In the present application, a cloud computing task is submitted to a Kubernetes cluster. A user usually submits a cloud computing batch task to the cluster, and the batch task includes a plurality of subtasks and specific information of each subtask. Then, the current cloud computing task to be scheduled is determined according to the specific information of the batch task.

[0051] In a specific embodiment, before the determining the current cloud computing task to be scheduled, the method further comprises: obtaining a cloud computing batch task; determining whether the cloud computing batch task requests a virtual neural network processor in the physical resource information, to obtain a first determination result; the virtual neural network processor is a processor obtained by virtualizing and dividing a physical neural network processor; if the first determination result indicates that the cloud computing batch task requests a virtual neural network processor in the physical resource information, determining whether the number of virtual neural network processors requested by the cloud computing batch task is greater than a preset number threshold, to obtain a second determination result; if the second determination result indicates that the number of virtual neural network processors requested by the cloud computing batch task is not greater than the preset number threshold, determining whether the memory value corresponding to the virtual neural network processor requested by the cloud computing batch task is greater than a preset memory threshold, to obtain a third determination result; if the third determination result indicates that the memory value corresponding to the virtual neural network processor requested by the cloud computing batch task is not greater than the preset memory threshold, determining the cloud computing batch task as a batch task to be scheduled, so as to determine the current cloud computing task to be scheduled from the batch task to be scheduled. Specifically, after obtaining the cloud computing batch task submitted by the user, the specific information of the task is checked. First, it is determined whether the physical resource information in the cluster contains the name of the virtual neural network processor requested by the cloud computing batch task. If not, the related cloud computing task does not need to be scheduled. Correspondingly, if the physical resource information of the cluster contains the name of the virtual neural network processor requested by the cloud computing batch task, the number of virtual neural network processors requested by the cloud computing batch task can be further determined. If the number of requests exceeds the corresponding limit, the task scheduling is also not needed. Further, if the number of virtual neural network processors requested by the cloud computing batch task does not exceed the preset number threshold, the memory value of the requested virtual neural network processor can be further determined. If the memory value of the request is greater than the preset memory threshold, the task scheduling is not needed. Otherwise, the cloud computing batch task meeting the condition can be determined as a batch task to be scheduled. It can be understood that the batch task to be scheduled contains a plurality of subtasks, i.e., a plurality of cloud computing tasks, which need to be independently scheduled and can be processed in parallel.

[0052] In another specific embodiment, the determining of the current cloud computing task to be scheduled can include: according to the number of virtual neural network processors and the display memory value in the virtual resource request corresponding to the cloud computing task, performing priority sorting on the subtasks in the obtained cloud computing batch task to obtain a priority queue corresponding to the cloud computing batch task; and performing iteration on the subtasks in the priority queue to determine the current subtask to be scheduled as the current cloud computing task. Specifically, for the obtained cloud computing batch task, the number of virtual neural network processors and the display memory value in the virtual resource request corresponding to each cloud computing task in the cloud computing batch task can be used to sort the cloud computing tasks in the cloud computing batch task to obtain a priority queue corresponding to the cloud computing batch task; for example, according to the product of the number of virtual neural network processors and the display memory value requested by the cloud computing task, the tasks are sorted according to the high and low values, the tasks requiring a lower amount of resources are moved forward, and finally the priority queue corresponding to the cloud computing batch task is obtained. Further, the subtasks in the priority queue can be iterated to determine the cloud computing task that needs to be scheduled at present. In specific embodiments, the priority flags of the subtasks in the cloud computing batch task can also be set in advance, and the task order is adjusted in combination with the priority flags to obtain the final priority queue.

[0053] Step S12: determining the candidate physical neural network processor corresponding to the current cloud computing task according to the matching relationship between the virtual resource request corresponding to the current cloud computing task and the physical resource information corresponding to the physical neural network processor in the current cluster.

[0054] In the present application, the current cloud computing task that needs to be scheduled and the virtual resource request corresponding to the task can be determined through the above steps; then the resource matching relationship between the physical resource information corresponding to the physical neural network processor in the current cluster and the virtual resource request corresponding to the current cloud computing task can be determined, and based on the resource matching relationship, the candidate physical neural network processor corresponding to the current cloud computing task in the current cluster can be further determined.

[0055] In a specific embodiment, the matching relationship between the virtual resource request corresponding to the current cloud computing task and the physical resource information corresponding to the physical neural network processor in the current cluster can comprise: determining the matching virtual device type from the physical resource information corresponding to the physical neural network processor in the current cluster according to the number and memory value of the virtual neural network processor requested by the current cloud computing task; the virtual device type is the device type of the virtual neural network processor obtained by virtualizing the physical neural network processor; and determining the physical neural network processor corresponding to the virtual device type as the candidate physical neural network processor corresponding to the current cloud computing task. Specifically, the virtual resource request corresponding to the current cloud computing task includes request information for the number and memory value of the virtual neural network processor; and the physical resource information corresponding to the physical neural network processor in the current cluster includes the device type of the virtual neural network processor obtained by virtualizing the physical neural network processor; according to the matching relationship between the virtual resource request and the device type of the virtual neural network processor obtained by virtualizing the physical neural network processor, a number of physical neural network processors suitable for the current cloud computing task can be determined from the physical resource information of the current cluster, i.e. the candidate physical neural network processor corresponding to the current cloud computing task is obtained. It can be understood that the device type of the virtual neural network processor created by a physical neural network processor is fixed, and the resources allocated to the virtual neural network processor are also fixed.

[0056] Step S13, based on a preset processor scoring rule, combining the virtual resource request, determining the target physical neural network processor corresponding to the current cloud computing task and the target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor.

[0057] In the present application, the candidate physical neural network processor matching the current cloud computing task can be determined from the physical resource information of the current cluster through the above steps; then the candidate physical neural network processor can be scored based on the preset processor scoring rule to determine the target physical neural network processor most suitable for the current cloud computing task, and the target physical neural network processor corresponds to a computing node, i.e. the target physical neural network processor and the target scheduling node corresponding to the current cloud computing task in the current cluster are finally determined.

[0058] In a specific embodiment, the determining, based on the preset processor scoring rule and in combination with the virtual resource request, the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors and a target scheduling node corresponding to the target physical neural network processor can include: scoring the candidate physical neural network processors by the preset processor scoring rule to obtain corresponding processor scores; determining the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors based on the virtual resource request, the processor scores, and a number of virtual neural network processors corresponding to the candidate physical neural network processors, and determining the target scheduling node corresponding to the target physical neural network processor. Specifically, the candidate physical neural network processors corresponding to the current cloud computing task can be scored by the preset processor scoring rule to obtain corresponding processor scores; the scoring can consider state information such as temperature, usage rate, and video memory of the processors; then, in combination with the number of virtual neural network processors corresponding to the candidate physical neural network processors, the related processor scores, and the virtual resource request of the current cloud computing task, the final target physical neural network processor and the corresponding target scheduling node are determined. In a specific embodiment, a part of the physical neural network processors can be filtered according to whether the virtual neural network processors are allocable, and then the target physical neural network processor corresponding to the current cloud computing task and the corresponding target scheduling node are determined in combination with the processor scores.

[0059] In another specific embodiment, the determining, based on the virtual resource request, the processor scores, and a number of virtual neural network processors corresponding to the candidate physical neural network processors, the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors can include: sorting the candidate physical neural network processors based on the processor scores from large to small to obtain a corresponding first list; and traversing the first list according to the number of virtual neural network processors corresponding to the virtual resource request to determine the target physical neural network processor corresponding to the current cloud computing task from the first list. Specifically, the candidate physical neural network processors can be sorted according to the processor scores to obtain a first list, which contains the number information of virtual neural network processors supported by the physical neural network processors; then, the target physical neural network processor for the current cloud computing task is determined according to the number of virtual neural network processors requested by the current cloud computing task. It can be understood that after the physical neural network processor is allocated for the current cloud computing task, the node to which the physical neural network processor belongs can be taken as the scheduling node of the cloud computing task.

[0060] In step S14, the target scheduling node creates virtual resource information corresponding to the virtual resource request, so as to execute the current cloud computing task by using the virtual resource information.

[0061] In the present application, the target physical neural network processor corresponding to the current cloud computing task and the corresponding target scheduling node can be determined from the physical resource information of the current cluster through the above steps; then, the virtual resource information corresponding to the virtual resource request of the current cloud computing task can be created by the target scheduling node, and the task pod (cluster scheduling unit) corresponding to the current cloud computing task in the node can be processed by the created virtual resource information.

[0062] In a specific embodiment, the step of creating, by the target scheduling node, virtual resource information corresponding to the virtual resource request, so as to execute the current cloud computing task by using the virtual resource information, can include: creating, by the target scheduling node, a target virtual neural network processor corresponding to the virtual resource request, and mounting the target virtual neural network processor to the current cloud computing task; and executing the current cloud computing task by using the target virtual neural network processor. Specifically, the target virtual neural network processor corresponding to the virtual resource request of the current cloud computing task can be created by the target scheduling node; in the Kubernetes cluster, the node can create an actual underlying virtual neural network processor (virtual device) by using the Ascend Docker Runtime (container runtime for managing and scheduling NPU resources), and randomly generate a device number; the task pod corresponding to the current cloud computing task can mount the actual virtual neural network processor to the internal container by using the device number, so as to complete the binding of the virtual device and the physical device. It can be understood that the Kubernetes cluster can start the task pod by using the Ascend Docker Runtime, and process the corresponding cloud computing task.

[0063] As can be seen, the present application can use virtualization technology to schedule and execute cloud computing tasks by using the virtual resource information corresponding to the physical neural network processor, so as to improve the resource utilization rate of the physical neural network processor; and by using the processor scoring rule, the virtual resource can be dynamically allocated according to the running state of the cluster, so as to improve the processing efficiency of the cloud computing task. In addition, for large-scale cloud computing batch tasks, the resources can be dynamically managed and adjusted in real time by using the resource scheduling and automatic expansion capabilities of the Kubernetes cluster, and each sub-task can be scheduled in parallel, so as to flexibly cope with the task demand and ensure that the task can obtain the required computing resources in time, thereby improving the task processing efficiency.

[0064] As shown in FIG. 1, the present application discloses a task scheduling method, which specifically includes: Figure 2 As shown in FIG. 1, the present application discloses a task scheduling method, which specifically includes:

[0065] In this embodiment, the Kubernetes cluster involves an AI batch task management module, an AI batch task checking module, an AI batch task scheduling module, and an AI batch task device allocation module. Specifically, a user can submit a batch task to the AI batch task management module, and the batch task needs to define the running image, start command, running environment variable, requested vNPU (Virtual Neural Processing Unit) device name, requested vNPU card number, and requested single vNPU memory amount of each subtask. After receiving the batch task, the AI batch task management module sends the batch task metadata information to the AI batch task processing module.

[0066] Then, the AI batch task checking module further processes the batch task submitted by the user and checks the resource request of the batch task to the vNPU. First, the requested vNPU device name is checked. If the node resource does not contain the vNPU device resource, the batch task creation is cancelled. Second, the vNPU request device number is checked. If the total number of vNPU virtual devices applied by the batch task exceeds the maximum limit, the batch task is also cancelled. Finally, the request amount of vNPU resources is checked. If the memory request amount of each vNPU exceeds the maximum limit of a single card, the batch task creation is also cancelled. If the vNPU resource request check is passed, the AI batch task processing module submits the user batch task to the AI batch task scheduler module.

[0067] Further, the AI batch task scheduling module allocates the Ascend NPU device and schedules the node for the subtasks in the batch task according to the physical Ascend NPU resources of the entire cluster and the virtual device segmentation types supported by the NPU card. Specifically, the AI batch task scheduling module includes an NPU scheduler and a Job (task) scheduler. The NPU scheduler allocates the available vNPU virtual device type and physical NPU device for each subtask in the current batch task according to the vNPU memory request value and the vNPU device quantity of the subtask. The vNPU virtual device is a bottom-layer segmentation of the NPU device, and it can be allocated part of the resources of the NPU physical device. The vNPU device type that can be created for an NPU device is fixed, and the resources allocated by the vNPU device type are also fixed. The Job scheduler further determines the final scheduling NPU device and computing node of each subtask in the batch task according to the score of each NPU device output by the NPU scheduler.

[0068] Specifically, the NPU scheduler can calculate the use priority of the NPU according to the AICore (core unit for performing AI calculation) utilization rate, AICPU utilization rate, vNPU virtual device assignable number, chip temperature, and use condition of the video memory of each NPU card. The AICore total amount and used amount, AICPU total amount and used amount, vNPU virtual device assignable number, chip temperature, total video memory value and used video memory value of the NPU device i can be collected in real time through the interface provided by the system self-provided NPU-smi function, and are respectively represented as , , , , , , , . The NPU scheduler dynamically calculates the Core i , CPU i , and M i of each NPU device by real-time sensing the use rate of the NPU node. The number of subtasks and the vNPU request video memory value of the subtasks in the batch task are obtained by analyzing the user request, the vNPU device type required by the subtask is calculated according to the vNPU request video memory value, the assignable vNPU virtual device number VDT i on the physical NPU device is obtained according to the vNPU device type, and in the scheduling, all NPU cards with no assignable virtual device are filtered out, that is, VDT i is 0, and the NPU cards with assignable vNPU devices are reserved.

[0069] In the reserved NPU devices, the real-time score of each NPU device is calculated to better reflect the availability of the NPU device i, and the comprehensive score S i is used to represent, as shown in the following formula:

[0070] ;

[0071] In the formula, the product of the AICore, AICPU, and video memory utilization rate of each NPU device is compared with the temperature of the NPU device to obtain the available score of each NPU device. The temperature of the NPU device can reflect its performance, and the lower the value is, the higher the available score of the device is.

[0072] Then, the ID (Identification, name) of the NPU device of all nodes, S i score, and VDT i virtual device number are saved to the list Q and provided for the Job scheduler to use.

[0073] Correspondingly, the Job scheduler can obtain the score list Q and the total amount of vNPU virtual devices required by the batch task, and then obtain four parameters of the number of each subtask in the batch task, the subtask priority flag, the required vNPU virtual device quantity and the vNPU device memory request quantity. The Job scheduler will sort the S i score list Q each time, and take out the record with the highest score first, obtain the VDT i virtual device quantity of the record, and subtract the quantity from the total amount of vNPU virtual devices required by the batch task, which represents that the NPU device can provide VDT i virtual devices for the batch task, and finally record the NPU device ID allocated for the batch task this time, the node to which the NPU device belongs and the VDT i virtual device quantity to the list A. Repeat the above steps until the total amount of vNPU virtual devices required by the batch task is 0.

[0074] The Job scheduler can then create a subtask priority queue P, and allocate vNPU resources to the subtasks with high priority first. When the priorities of each subtask in the batch task are different, and the requested vNPU resources are also different, the Job scheduler will allocate the NPU device with the optimal score and the node to which the NPU device belongs to the subtask with the highest priority. The priority sorting process is as follows: 1. Calculate the product of the vNPU virtual device quantity and the vNPU device memory quantity requested by each subtask. When the product of the vNPU virtual device quantity and the vNPU device memory quantity of a certain subtask is lower than that of other subtasks, the subtask requiring a lower resource quantity is moved forward. Finally, a temporary subtask priority queue F is obtained. 2. Sort the subtasks in the F queue according to the priority flag. If a subtask has a higher priority flag, the subtask is moved to the front end of F to wait for resource allocation. The queue order obtained finally is the priority queue P. It can be understood that the task with a smaller product of the vNPU device quantity and the memory value is processed preferentially, and the subtask with the priority flag can be allocated resources preferentially under the same conditions.

[0075] It can be understood that each record in the list A contains the maximum vNPU virtual device quantity that each NPU device can provide for the batch task. At this time, each subtask in the subtask priority queue P has not been allocated a specific NPU device ID and a running node, so it is necessary to obtain all the subtasks in the subtask priority queue P, obtain each record by traversing the list A, and bind the NPU device ID and the node to which the device belongs in the record to each subtask according to the vNPU virtual device quantity required by the subtask. At the same time, the VDT i virtual device quantity in the record is matched with the vNPU device quantity required by the subtask. The matching process is as follows: 1. If the vNPU device quantity required by the task is greater than the VDT ithe VDT of the record is directly assigned to the task i the VDT of the record is directly assigned to the task i the VDT of the record is directly assigned to the task i the VDT of the record is directly assigned to the task i the VDT of the record is directly assigned to the task i the VDT of the record is directly assigned to the task

[0076] It should be noted that a resource buffer pool can be reserved in the Job scheduler to allocate resources for high-priority tasks. A resource allocation threshold can also be set in advance, and when the remaining resources are lower than the threshold, load balancing is triggered to avoid some NPUs running overloaded due to high task load, while reducing the idle time of other NPUs to extend the service life of the device and reduce maintenance costs. In addition, resource preemption mechanisms can be supported between cloud computing tasks to optimize resource allocation.

[0077] Further, each sub-task Pod in the batch task has been assigned a specific NPU physical device ID and a scheduling node through the AI batch task scheduling module. At this time, the Kubelet process on the scheduling node sends a vNPU device request to the AI batch task device allocation module. The AI batch task device allocation module obtains all sub-task Pods in the Pending state on the current node and processes these sub-tasks one by one. The main process is as follows: 1. According to the physical NPU device, vNPU device type and vNPU device request quantity set by the scheduler module for the sub-task Pod, the device type request is converted into the virtual NPU device template name supported by the Ascend card according to the vNPU template, and the device template name and vNPU device request quantity are transmitted to the Ascend Docker Runtime to create the actual underlying vNPU virtual device, and the virtual device type is mediated. The virtual device randomly generates a UUID (Universally Unique Identifier) device number, and the sub-task Pod mounts the actual VNPU virtual device to its internal container through the UUID. 2. After creating the vNPU virtual device, the AI batch task scheduler module returns the UUID of all virtual devices to the Kubelet, and the Kubelet starts the sub-task Pod using the Ascend Docker Runtime. After the sub-task Pod is in the Running state, it can normally use the vNPU virtual device. 3. After waiting for all sub-task Pods in the batch task to be in the Running state, the batch task creation success information is returned. In this way, according to the actual needs of the AI batch task, dynamic resource allocation is performed through the virtual Ascend NPU device, so that each Ascend NPU device can be fully utilized, thereby greatly improving the overall utilization rate of NPU resources.

[0078] It can be seen that in the scheme, the Ascend NPU device is finely divided by introducing virtualization technology to form multiple virtual Ascend NPU devices, and is dynamically allocated according to the actual needs of the AI batch task. In this way, the problem of resource idling and waste caused by static allocation can be effectively avoided, and each Ascend NPU device can be fully utilized, thereby greatly improving the overall utilization rate of NPU resources. It can be understood that in a large-scale AI batch task processing scenario, the virtual Ascend NPU resources can be reasonably allocated according to the dependency relationship and resource demand of the task to ensure that each subtask can run in parallel. This not only improves the efficiency of task processing, but also shortens the task completion time, meeting the urgent demand of large-scale AI applications for high-performance computing NPU resources. It should be pointed out that compared with the traditional static allocation method, the scheme can utilize the powerful resource scheduling and automatic expansion capability of the Kubernetes platform to realize dynamic management and real-time adjustment of the Ascend NPU resources. In this way, the dynamic changes in AI task demand can be flexibly responded to, ensuring that tasks can obtain the required computing resources in real time, improving the flexibility and response speed of resource scheduling. Further, the present application comprehensively considers the AI parallel task demand, NPU resource state, AI task priority and dynamic changes of the cluster, and realizes more intelligent and efficient Ascend NPU resource allocation and task scheduling. This helps to optimize the load balancing of the cluster, avoids overloading of some NPU due to high task load, and reduces the idle time of other NPU, thereby prolonging the service life of the device and reducing the maintenance cost.

[0079] As shown in Figure 3 The embodiment of the application discloses a task scheduling device applied to a Kubernetes cluster, which comprises:

[0080] The task determination module 11 is configured to determine a current cloud computing task to be scheduled;

[0081] The first processor determination module 12 is configured to determine a candidate physical neural network processor corresponding to the current cloud computing task according to the matching relationship between the virtual resource request corresponding to the current cloud computing task and the physical resource information corresponding to the physical neural network processor in the current cluster;

[0082] The second processor determination module 13 is configured to determine a target physical neural network processor corresponding to the current cloud computing task and a target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request;

[0083] The task execution module 14 is configured to create virtual resource information corresponding to the virtual resource request through the target scheduling node, so as to execute the current cloud computing task by using the virtual resource information.

[0084] Therefore, the cloud computing task can be scheduled and executed by means of the virtualization technology and the physical neural network processor corresponding to the virtual resource information, so that the resource utilization rate of the physical neural network processor can be improved. In combination with the processor scoring rule, the virtual resource can be dynamically allocated according to the running condition of the cluster, so that the processing efficiency of the cloud computing task can be improved.

[0085] In a specific embodiment, the task determination module 11 can include:

[0086] The task sorting unit is configured to perform priority sorting on the subtasks in the obtained cloud computing batch task according to the number of virtual neural network processors and the video memory value in the virtual resource request corresponding to the cloud computing task, to obtain a priority queue corresponding to the cloud computing batch task.

[0087] The task determination unit is configured to traverse the subtasks in the priority queue to determine the current subtask to be scheduled as the current cloud computing task.

[0088] In a specific embodiment, the first processor determination module 12 can include:

[0089] The virtual device type determination unit is configured to determine a matching virtual device type from the physical resource information corresponding to the physical neural network processor in the current cluster according to the number of virtual neural network processors and the video memory value requested by the current cloud computing task; the virtual device type is a device type corresponding to the virtual neural network processor obtained by virtualizing and cutting the physical neural network processor.

[0090] The first processor determination unit is configured to determine the physical neural network processor corresponding to the virtual device type as the candidate physical neural network processor corresponding to the current cloud computing task.

[0091] In a specific embodiment, the second processor determination module 13 can include:

[0092] The processor scoring unit is configured to score the candidate physical neural network processor by using a preset processor scoring rule, to obtain a corresponding processor score.

[0093] The second processor determining sub-module is configured to determine a target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors based on the virtual resource request, the processor score, and a number of virtual neural network processors corresponding to the candidate physical neural network processor, and determine a target scheduling node corresponding to the target physical neural network processor.

[0094] In another specific embodiment, the second processor determining sub-module can include:

[0095] The processor sorting unit is configured to sort the candidate physical neural network processors based on the processor scores from large to small to obtain a corresponding first list.

[0096] The second processor determining unit is configured to traverse the first list according to the number of virtual neural network processors corresponding to the virtual resource request to determine a target physical neural network processor corresponding to the current cloud computing task from the first list.

[0097] In one specific embodiment, the task execution module 14 can include:

[0098] The task mounting unit is configured to create a target virtual neural network processor corresponding to the virtual resource request through the target scheduling node, and mount the target virtual neural network processor to the current cloud computing task.

[0099] The task execution unit is configured to execute the current cloud computing task by using the target virtual neural network processor.

[0100] In one specific embodiment, the apparatus can further include:

[0101] The task obtaining module is configured to obtain a cloud computing batch task.

[0102] The first judgment module is configured to determine whether there is a name of a virtual neural network processor requested by the cloud computing batch task in the physical resource information to obtain a corresponding first judgment result; the virtual neural network processor is a processor obtained by virtualizing and dividing a physical neural network processor.

[0103] The second judgment module is configured to, when the first judgment result indicates that there is the name of the corresponding virtual neural network processor in the physical resource information, determine whether a number of virtual neural network processors requested by the cloud computing batch task is greater than a preset number threshold to obtain a corresponding second judgment result.

[0104] The third determining module is configured to determine whether the value of the video memory corresponding to the virtual neural network processor of the cloud computing batch task request is greater than a preset video memory threshold, to obtain a corresponding third determining result, when the second determining result indicates that the number of the virtual neural network processor of the cloud computing batch task request is not greater than a preset number threshold.

[0105] The batch task determining module is configured to determine the cloud computing batch task as a to-be-scheduled batch task, when the third determining result indicates that the value of the video memory corresponding to the virtual neural network processor of the cloud computing batch task request is not greater than the preset video memory threshold, so as to determine a to-be-scheduled current cloud computing task from the to-be-scheduled batch task.

[0106] Further, the embodiment of the present application further discloses an electronic device, Figure 4 The electronic device 20 shown in the figure is not considered as any limitation on the use range of the present application.

[0107] Figure 4 The electronic device 20 provided by the embodiment of the present application is shown in the structural schematic diagram of the electronic device 20. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is used to store a computer program, the computer program is loaded and executed by the processor 21, to realize the related steps in the task scheduling method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0108] In the embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0109] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0110] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. The computer program 222 can further include computer programs capable of performing other specific tasks in addition to the computer programs capable of performing the task scheduling method performed by the electronic device 20 disclosed in any of the foregoing embodiments.

[0111] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the task scheduling method disclosed above. For the specific steps of the method, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0112] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0113] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in general in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0114] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0115] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the stated element.

[0116] The above detailed description of the technical solutions provided by the present application has been provided, and the principles and implementation modes of the present application have been described by applying specific examples. The above description of the examples is only for the purpose of helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. A task scheduling method, characterized by, The application is applied to a Kubernetes cluster, and comprises: determining a current cloud computing task to be scheduled; determining a candidate physical neural network processor corresponding to the current cloud computing task according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to a physical neural network processor in a current cluster; the physical resource information comprises a virtual device type corresponding to a virtual neural network processor after virtualization and segmentation of each physical neural network processor; determining a target physical neural network processor corresponding to the current cloud computing task and a target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor based on a preset processor scoring rule and in combination with the virtual resource request; creating virtual resource information corresponding to the virtual resource request through the target scheduling node, so as to execute the current cloud computing task by using the virtual resource information; wherein the determining of the candidate physical neural network processor corresponding to the current cloud computing task according to the matching relationship between the virtual resource request corresponding to the current cloud computing task and the physical resource information corresponding to the physical neural network processor in the current cluster comprises: determining a matching virtual device type from the physical resource information corresponding to the physical neural network processor in the current cluster according to a quantity of virtual neural network processors and a display memory value requested by the current cloud computing task; the device type of the physical neural network processor and the virtual neural network processor is a fixed corresponding relationship; determining the physical neural network processor corresponding to the virtual device type as the candidate physical neural network processor corresponding to the current cloud computing task.

2. The task scheduling method of claim 1, wherein, The determining of the current cloud computing task to be scheduled comprises: performing priority sorting on subtasks in a cloud computing batch task obtained according to a quantity of virtual neural network processors and a display memory value in a virtual resource request corresponding to a cloud computing task, to obtain a priority queue corresponding to the cloud computing batch task; determining a current cloud computing task to be scheduled by traversing the subtasks in the priority queue.

3. The task scheduling method of claim 1, wherein, The determining of the target physical neural network processor corresponding to the current cloud computing task and the target scheduling node corresponding to the target physical neural network processor from the candidate physical neural network processor based on the preset processor scoring rule and in combination with the virtual resource request comprises: scoring the candidate physical neural network processor by using a preset processor scoring rule to obtain a corresponding processor score; determining the target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processor based on the virtual resource request, the processor score and a quantity of virtual neural network processors corresponding to the candidate physical neural network processor, and determining the target scheduling node corresponding to the target physical neural network processor.

4. The task scheduling method of claim 3, wherein, The determining, based on the virtual resource request, the processor score, and the number of virtual neural network processors corresponding to the candidate physical neural network processor, a target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processor, includes: ordering the candidate physical neural network processors based on the processor score from large to small to obtain a corresponding first list; traversing the first list according to the number of virtual neural network processors corresponding to the virtual resource request to determine the target physical neural network processor corresponding to the current cloud computing task from the first list.

5. The task scheduling method of claim 1, wherein, The creating, by the target scheduling node, virtual resource information corresponding to the virtual resource request to execute the current cloud computing task by using the virtual resource information includes: creating, by the target scheduling node, a target virtual neural network processor corresponding to the virtual resource request, and mounting the target virtual neural network processor to the current cloud computing task; executing the current cloud computing task by using the target virtual neural network processor.

6. The task scheduling method according to any one of claims 1 to 5, characterized in that, Before the determining the current cloud computing task to be scheduled, the method further includes: obtaining a cloud computing batch task; determining whether the name of the virtual neural network processor requested by the cloud computing batch task exists in the physical resource information to obtain a corresponding first determination result; the virtual neural network processor is a processor obtained by virtualizing and dividing a physical neural network processor; if the first determination result indicates that the name of the corresponding virtual neural network processor exists in the physical resource information, determining whether the number of virtual neural network processors requested by the cloud computing batch task is greater than a preset number threshold to obtain a corresponding second determination result; if the second determination result indicates that the number of virtual neural network processors requested by the cloud computing batch task is not greater than the preset number threshold, determining whether the value of the display memory corresponding to the virtual neural network processor requested by the cloud computing batch task is greater than a preset display memory threshold to obtain a corresponding third determination result; if the third determination result indicates that the value of the display memory corresponding to the virtual neural network processor requested by the cloud computing batch task is not greater than the preset display memory threshold, determining the cloud computing batch task as a batch task to be scheduled to determine the current cloud computing task to be scheduled from the batch task to be scheduled.

7. A task scheduling apparatus characterized by comprising: The application is applied to a Kubernetes cluster, and includes: a task determination module configured to determine a current cloud computing task to be scheduled; a first processor determination module configured to determine, according to a matching relationship between a virtual resource request corresponding to the current cloud computing task and physical resource information corresponding to a physical neural network processor in a current cluster, a candidate physical neural network processor corresponding to the current cloud computing task; the physical resource information includes a virtual device type corresponding to a virtual neural network processor obtained by virtualizing and dividing each physical neural network processor; The second processor determining module is configured to determine, based on a preset processor scoring rule and in combination with the virtual resource request, a target physical neural network processor corresponding to the current cloud computing task from the candidate physical neural network processors and a target scheduling node corresponding to the target physical neural network processor; The task execution module is configured to create, through the target scheduling node, virtual resource information corresponding to the virtual resource request, so as to execute the current cloud computing task by using the virtual resource information; The first processor determining module comprises: The virtual device type determining unit is configured to determine, from physical resource information of physical neural network processors in a current cluster, a matching virtual device type according to a quantity of virtual neural network processors and a display memory value requested by the current cloud computing task; the device types of the physical neural network processors and the virtual neural network processors are in a fixed corresponding relationship; The first processor determining unit is configured to determine, as candidate physical neural network processors corresponding to the current cloud computing task, physical neural network processors corresponding to the virtual device type.

8. An electronic device, comprising: The computer program is stored in the memory and executed by the processor to implement the task scheduling method according to any one of claims 1 to 6. The computer program is stored in the memory and executed by the processor to implement the task scheduling method according to any one of claims 1 to 6. ​ 9. A computer-readable storage medium, characterized in that, ​

Images