Multi-acceleration task scheduling method and system for DPU architecture

Abstract

The invention discloses a multi-acceleration task scheduling method and system for a DPU architecture, and the method comprises the steps: generating a set of acceleration tasks to be scheduled according to a task submission timestamp sequence, and constructing a task pool composed of acceleration tasks in various states; performing polling management on the acceleration tasks in the task pool, executing the acceleration tasks in neutral and suspended states for each to-be-scheduled acceleration task, and respectively executing different task management strategies; and performing memory centralized scheduling management on the acceleration task between the local memory resource and the remote memory resource by adopting a memory balance strategy according to the local memory resource sufficiency condition and the remote memory resource access quantity condition. According to the method and the system, the hardware resource utilization rate of the DPU architecture is improved, the calculation efficiency and the storage efficiency of the DPU system are both considered, and various hardware resources applied to the DPU architecture of the data center can be fully exerted.

Description

Technical Field

[0001] The present application relates to the field of task scheduling technology, and in particular to a multi-acceleration task scheduling method and system for a DPU architecture. Background Art

[0002] At present, the emergence and development of emerging technologies such as big data analysis, artificial intelligence, and privacy computing have put forward higher demands on computer computing power. Traditional general computing architecture is difficult to provide the required computing power due to the constraints of chip area and computing efficiency. The dedicated domain architecture (Domain Specific Architecture, DSA) achieves computing acceleration for specific tasks by designing a special accelerator structure. Among them, the DPU (Data Processing Unit) smart network card has attracted widespread attention for its rich and secure hardware offloading capabilities.

[0003] The current multi-task scheduling scenarios are concentrated on general computing architectures, that is, considering the offloading of multiple tasks on general computing resources. The limitation of this technology is that general computing resources are relatively limited. In order to consider versatility, general computing architectures often have more complex control logic and CPU micro-architectures. Under certain area constraints, the logic resources used for data calculation are relatively scarce. Compared with acceleration in special fields, the effect of multi-task scheduling on general computing resources is relatively limited.

[0004] The dedicated domain acceleration architecture realizes the computational acceleration for specific tasks. The dedicated domain acceleration architecture provides sufficient computing resources, performs domain-specific architecture design for specific tasks, and fully utilizes hardware resources to achieve high parallelism in data processing. In this scenario, the allocation of computing resources is often no longer a bottleneck for a considerable number of tasks. With the advent of the big data era, the scale of data has grown exponentially, and the bottleneck of task acceleration has shifted from the shortage of computing resources to the shortage of storage resources. On the other hand, the growth of computing speed has far exceeded the growth of memory access speed in the past few decades. This phenomenon is called the "scissors gap" between computing speed and memory access speed. This "scissors gap" phenomenon further exacerbates the contradiction between computing and memory access in task acceleration, resulting in a situation where memory access cannot meet computing needs.

[0005] In response to the above two problems, the solutions in the DPU architecture are RDMA and hierarchical storage. RDMA technology uses memory pools to expand storage resources, and hierarchical storage is used to reduce memory access latency. However, when facing multiple acceleration tasks in parallel, the existing RDMA technology often simply uses RDMA to frequently swap pages in and out due to insufficient local memory resources, which increases memory access latency and seriously affects the execution efficiency of multiple acceleration tasks.

[0006] Therefore, the RDMA and hierarchical storage solutions in the DPU architecture urgently need a multi-acceleration task scheduling system that takes into account both computing efficiency and memory access efficiency. Summary of the invention

[0007] In order to solve the problems existing in the multi-acceleration task scheduling technology of the above-mentioned prior art, a multi-acceleration task scheduling method and system in a DPU architecture are proposed. The multi-acceleration task scheduling system takes into account the computing efficiency and memory access efficiency of multi-acceleration tasks, and provides a user-defined scheduling interface, further improving the flexibility of the multi-acceleration task system.

[0008] In a first aspect, an embodiment of the present application provides a multi-acceleration task scheduling method for a DPU architecture, wherein the DPU architecture includes multiple acceleration units, and the method includes:

[0009] The step of generating an acceleration task set is as follows: a set of acceleration tasks to be scheduled is generated in the order of task submission timestamps, and a task pool consisting of acceleration tasks in various states is constructed, wherein each acceleration task contains information on the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, executing, and suspended;

[0010] Task management step: poll the acceleration tasks in the task pool and execute different task management strategies for each acceleration task in the to-be-scheduled, executing, and suspended states;

[0011] Memory management steps: Based on the availability of local memory resources and the amount of access to remote memory resources, a memory balancing strategy is adopted to centrally schedule and manage the memory of acceleration tasks between local memory resources and remote memory resources.

[0012] In a specific embodiment of the present invention, the pending scheduling state of the above-mentioned acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task is suspended due to multiplexing with the same type of acceleration tasks on the same acceleration unit.

[0013] In a specific embodiment of the present invention, the above task management step further includes:

[0014] If the acceleration task is in the waiting-to-be-scheduled state, the task is assigned to different acceleration unit task queues according to the corresponding acceleration unit type, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in the waiting-to-be-scheduled state;

[0015] If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm;

[0016] If the task is in the suspended state, determine whether the execution conditions are met. If the execution conditions are met, execute the acceleration task and set the state to executing; otherwise, keep the acceleration task state as suspended.

[0017] In a specific embodiment of the present invention, the above memory management step further includes:

[0018] For acceleration tasks in the waiting-to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration tasks; otherwise, the current acceleration tasks are set to the suspended state;

[0019] For acceleration tasks in the execution state, if the storage request is determined to be an application for local memory resources, local memory resources will be allocated first if local memory resources are sufficient. Otherwise, the corresponding remote storage resources will be allocated to the storage request. If the storage request is determined to be an access to remote memory resources, if the remote memory access frequency exceeds the set threshold, the page will be marked as a hot page and replaced with a cold page in the local memory resource.

[0020] For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

[0021] In a second aspect, an embodiment of the present application provides a multi-acceleration task scheduling system for a DPU architecture, which adopts the multi-acceleration task scheduling method for a DPU architecture as described above, wherein the DPU architecture includes multiple acceleration units, and the system includes:

[0022] Acceleration task set generation module: Generates a set of acceleration tasks to be scheduled in the order of task submission timestamps, and builds a task pool consisting of acceleration tasks in various states, where each acceleration task contains the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, executing, and suspended;

[0023] Task management module: performs polling management on the acceleration tasks in the task pool, and executes different task management strategies for the acceleration tasks in the waiting, executing and suspended states;

[0024] Memory management module: Based on the availability of local memory resources and the access to remote memory resources, a memory balancing strategy is adopted to centrally manage the memory of all acceleration tasks in local memory resources or remote memory resources.

[0025] In a specific embodiment of the present invention, the pending scheduling state of the above-mentioned acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task and the same type of acceleration tasks are suspended on the same acceleration unit due to task multiplexing.

[0026] In a specific embodiment of the present invention, the task management module further includes:

[0027] Poll the acceleration tasks in the task pool. If the task is in the waiting-to-be-scheduled state, the task is assigned to different acceleration unit task queues according to the acceleration unit type of the acceleration task, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in the waiting-to-be-scheduled state;

[0028] If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm;

[0029] If the task is in the suspended state, determine whether the execution conditions are met. If the execution conditions are met, execute the acceleration task and set the state to executing; otherwise, keep the acceleration task state as suspended.

[0030] In a specific embodiment of the present invention, the memory management module further includes:

[0031] For acceleration tasks in the waiting-to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration tasks; otherwise, the current acceleration tasks are set to the suspended state;

[0032] For acceleration tasks in the executing state, if the storage request is determined to be an application for local memory resources, local memory resources will be allocated first if local memory resources are sufficient, otherwise, the corresponding remote storage resources will be allocated to the request;

[0033] When it is determined that the storage request is to access the remote memory resource, and the remote memory access frequency exceeds the set threshold, the page is marked as a hot page and replaced with a cold page in the local memory resource;

[0034] For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned multi-acceleration task scheduling method for a DPU architecture.

[0036] In a fourth aspect, an embodiment of the present application provides a DPU, comprising a memory, a dedicated acceleration unit, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the multi-acceleration task scheduling method for the DPU architecture as described above are implemented.

[0037] Compared with the related prior art, it has the following outstanding beneficial effects:

[0038] 1) The method of the present invention proposes a multi-acceleration task scheduling system that takes into account both computing efficiency and memory access efficiency in a DPU architecture. In view of the problem of shortage of local storage resources in the scenario of multiple acceleration tasks in parallel, the memory capacity is expanded by using a separate memory based on the RDMA protocol, and a remote memory storage layer is added on the basis of the existing storage layer. The separate memory solution can better alleviate the "scissors gap" caused by computing and memory access, and provides an acceptable memory access delay while expanding the available storage capacity of the acceleration task, thereby improving the execution efficiency of the acceleration task. In addition, from the perspective of the data center, the technology balances the memory usage of each node and improves the storage resource utilization of the data center.

[0039] 2) The method of the present invention aims at the problem of low utilization efficiency of computing resources and storage resources caused by the scheduling method in the scenario of multiple acceleration tasks in parallel. The present invention provides an efficient comprehensive task scheduling system that takes into account both the scheduling of tasks of the same type and the scheduling of tasks of different types; thereby improving the utilization efficiency of computing resources and storage resources in the DPU architecture and further improving the hardware offloading capability of the DPU;

[0040] 3) The method of the present invention provides a user-configurable interface for potential user-defined needs in scenarios where multiple acceleration tasks are carried out in parallel, thereby improving the configurability and flexibility of the multi-acceleration task scheduling system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 This is a typical DPU architecture diagram;

[0043] Figure 2 This is a schematic diagram of a typical multi-acceleration task scheduling scenario;

[0044] Figure 3 A schematic diagram of a multi-acceleration task scheduling method for a DPU architecture according to the present invention;

[0045] Figure 4 The acceleration task state diagram of the present invention;

[0046] Figure 5 It is a flow chart of the task pool management algorithm of the present invention;

[0047] Figure 6 This is a flow chart of the memory management algorithm of the present invention;

[0048] Figure 7 Schematic diagram of a multi-acceleration task scheduling system for a DPU architecture according to the present invention. DETAILED DESCRIPTION

[0049] It should be noted that the processor described in the present invention is the control center of the electronic device, which may be a processor or a general term for multiple processing elements. For example, it may be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0050] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0051] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single core processor or a multi-core processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0052] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0053] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0054] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0055] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0056] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0057] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0058] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0059] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0060] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0061] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0062] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0063] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0064] The present invention aims to propose a multi-acceleration task scheduling method and system in a DPU architecture. The multi-acceleration task scheduling system takes into account the computing efficiency and memory access efficiency of the multi-acceleration tasks, and provides a user-defined scheduling interface, further improving the flexibility of the multi-acceleration task system.

[0065] When studying the multi-acceleration task system in the DPU architecture, the present invention found that the defect in the prior art is caused by only considering the allocation of computing resources but not considering the scissors difference contradiction between computing resources and memory access resources. This may be because it has not been noticed that the shortage of computing resources has been relatively alleviated in the process of transforming from a general computing architecture to a special-purpose acceleration architecture, while the shortage of local storage resources and the existence of memory access delay have become the new main contradiction. Through the study of the memory access characteristics of multiple acceleration tasks and the storage hierarchy with the addition of RDMA technology, the present invention found that solving this defect can be achieved by designing a method for a multi-acceleration task system.

[0066] like Figure 1 As shown in the figure, a typical DPU architecture includes a general-purpose computing unit CPU, dedicated acceleration units (such as vector processing units, machine learning acceleration units, privacy computing acceleration units, etc.), I / O modules, etc. DPU contains a variety of dedicated acceleration units, which can offload and hardware accelerate different tasks, such as vector processing, homomorphic encryption, neural network reasoning, video data processing, etc. In order to make full use of hardware resources, there are often multiple tasks of the same or different types that are offloaded to the same or different hardware accelerators at the same time. These tasks often have the characteristics of high memory access, that is, there are frequent and large memory access requirements.

[0067] like Figure 2As shown in the figure, a typical multi-acceleration task scheduling scenario, the task pool is the tasks to be scheduled abstracted from the system level. These tasks have their own timestamp information according to the submission order. Secondly, the task type determines the type of acceleration unit to which the task is assigned; the computing resource pool is the acceleration unit resource abstracted from the hardware architecture level. In a heterogeneous computing system, there are often multiple acceleration units responsible for accelerating different task types; the memory resource pool is the storage resource abstracted from the system storage level, which is divided into local memory resources and remote memory resources. The delay of accessing local storage resources is less than the delay of accessing remote memory resources. The following conclusions can be drawn from the analysis of this multi-acceleration task scheduling scenario: 1. From the perspective of task type, multi-acceleration task scheduling adds the dimension of task type to the scenario of traditional kernel task scheduling. It is necessary to consider both the scheduling of the same task type on the same acceleration unit and the scheduling of different task types on different acceleration units. 2. From the perspective of task scheduling goals, multi-acceleration task scheduling highlights the bottleneck brought by memory access in the scenario of traditional kernel task scheduling.

[0068] Due to the limited local storage resources, Disaggregated Memory uses RDMA (Remote Direct Memory Access) technology to integrate local and remote memory resources into a memory pool. When the local memory cannot accommodate the amount of data that the application needs to access, remote memory resources can be flexibly used through network access and other methods to alleviate the problem of insufficient local memory resources. Disaggregated Memory using RDMA technology not only alleviates the problem of insufficient local memory resources, but also balances the memory resource utilization in the computer cluster and enhances the stability of the cluster.

[0069] RDMA technology allows a server node to directly access the memory of another server node at a remote end. In a data center, if a server node can only use the memory of its own node, due to the differences in the number and type of load tasks carried by different server nodes, the memory resource usage of different servers will be quite different (i.e., some server nodes may have a high memory occupancy rate while some server nodes may have a low memory occupancy rate), thus showing the phenomenon of uneven memory utilization of server nodes in the data center, which will greatly affect the execution performance of load tasks. Using RDMA technology, a server node can directly access the memory of another server node at a remote end, thereby improving the execution efficiency of accelerated tasks and improving the utilization of storage resources in the data center. The time required for a node to access local storage resources (faster) and access remote memory resources (slower) through RDMA technology is different, so the key point of the present invention is how to balance the execution speed of multiple accelerated tasks and the memory utilization efficiency of the data center through a memory scheduling algorithm.

[0070] The goal of the multi-acceleration task scheduling of the present invention is to focus on the efficient use of storage resources on the basis of making full use of computing resources. On the one hand, tasks are reasonably scheduled to improve the performance of the entire system; on the other hand, load balancing is achieved in the multi-acceleration task scenario to enhance the stability and robustness of the entire system. At present, the perspective of multi-task scheduling technical solutions is mostly focused on the general computing architecture, and more consideration is given to the problem of computing resource allocation. The scenario considered by multi-acceleration tasks is how to alleviate the memory access bottleneck problem caused by frequent and large memory access requirements when computing resources are relatively sufficient.

[0071] The following is a detailed description with reference to specific embodiments:

[0072] Embodiment 1

[0073] like Figure 3 As shown, the embodiment of the present application provides a multi-acceleration task scheduling method for a DPU architecture, the DPU architecture includes multiple acceleration units, and the method includes:

[0074] Acceleration task set generation step 101: Generate a set of acceleration tasks to be scheduled in the order of task submission timestamps, and build a task pool consisting of acceleration tasks in various states, wherein each acceleration task contains information about the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, executing, and suspended;

[0075] Task management step 102: performing polling management on the acceleration tasks in the task pool, and executing different task management strategies for each acceleration task in the to-be-scheduled, executing, and suspended states;

[0076] Memory management step 103: Based on the sufficiency of local memory resources and the access volume of remote memory resources, a memory balancing strategy is adopted to perform centralized memory scheduling management for acceleration tasks between local memory resources and remote memory resources.

[0077] Among them, the pending scheduling state of the above-mentioned acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task is suspended due to reuse with the same type of acceleration tasks on the same acceleration unit.

[0078] In a specific embodiment of the present invention, in order to solve the problem of shortage of local storage resources in the scenario of multiple acceleration tasks in parallel, the memory capacity is expanded by using the separated memory based on the RDMA protocol, and a remote memory storage layer is added on the basis of the existing storage layer; the offload tasks from the upper layer undertaken by the DPU are often memory-intensive, and the acceleration unit alleviates the bottleneck of computing resource shortage and also highlights the memory access bottleneck in the acceleration task. Since the storage capacity and storage delay are affected by the development of storage media and develop relatively slowly, the separated memory solution can better alleviate the "scissors gap" caused by computing and memory access, and provides an acceptable memory access delay while expanding the available storage capacity of the acceleration task, thereby improving the execution efficiency of the acceleration task. In addition, from the perspective of the data center, the technical solution of the present invention balances the memory usage of each node and improves the storage resource utilization of the data center;

[0079] In a specific embodiment of the present invention, in order to solve the problem of low utilization efficiency of computing resources and storage resources caused by the scheduling method in the scenario of multiple acceleration tasks in parallel, the present invention provides an efficient comprehensive task scheduling system that takes into account both the scheduling of tasks of the same type and the scheduling of tasks of different types; the present invention develops an efficient multi-acceleration task scheduling system in the DPU architecture and needs to consider two scheduling levels: 1) The same type of acceleration task scheduling level. Tasks of the same type refer to acceleration tasks scheduled to the same acceleration unit. The goal of scheduling the same type of acceleration tasks is to make full use of the hardware computing resources of the acceleration unit, and the same type of acceleration tasks have certain similarities in memory access mode. 2) Different types of acceleration task scheduling level. Different types of acceleration task scheduling refers to acceleration tasks scheduled to different acceleration units. The goal of scheduling different types of acceleration tasks is to reduce the average memory access delay and increase the utilization efficiency of storage resources by improving the utilization efficiency of local storage resources and minimizing the frequency of remote memory access. In summary, the scheduling system of the present invention can take into account the scheduling of acceleration tasks of the same type and the scheduling of acceleration tasks of different types, thereby improving the utilization efficiency of computing resources and storage resources in the DPU architecture, and further improving the hardware offloading capability of the DPU;

[0080] In a specific embodiment of the present invention, a user-customizable interface is provided for potential user-customized needs in scenarios where multiple acceleration tasks are carried out in parallel. The starting point of the design of the multi-acceleration task scheduling system in the DPU architecture is to make full use of computing resources and storage resources, but the end point is often the user experience. Most design principles are more consistent with the user experience, such as reducing the execution time of the acceleration task, etc. However, sometimes users may have customized scheduling requirements, such as the priority and privacy requirements of different user task processing. In order to improve the configurability and flexibility of the multi-acceleration task scheduling system, an interface for user-customized scheduling configuration is designed in the system.

[0081] Embodiment 2

[0082] The embodiment of the present application provides a multi-acceleration task scheduling method for a DPU architecture, wherein the DPU architecture includes multiple acceleration units, and the method includes:

[0083] Acceleration task set generation step 201: Generate a set of acceleration tasks to be scheduled in the order of task submission timestamps, and build a task pool consisting of acceleration tasks in various states, wherein each acceleration task contains information about the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, executing, and suspended;

[0084] There are different types of accelerated tasks in the task pool. Different types of tasks can only be scheduled to the corresponding accelerator for accelerated execution, such as Figure 1 As shown in the figure, deep learning tasks can only be scheduled to the machine learning acceleration unit for acceleration, and privacy computing tasks can only be scheduled to the privacy computing acceleration unit for acceleration; secondly, there is often more than one acceleration task of the same type in the task pool, such as Figure 2 As shown, the same type of tasks are identified by the same task, and the type 1 tasks include at least two tasks, namely task 1 and task 4, and the type 3 tasks include at least two tasks, namely task 3 and task 5. In a specific embodiment, the scheduling of acceleration tasks of the same type and different types is reflected in the task pool management algorithm uniformly coarsening all tasks regardless of their types, which is the scheduling of acceleration tasks of different types. For tasks in the state of being scheduled, the tasks are placed in different scheduling queues according to their types for more fine-grained task management. For example, the same type of tasks will record historical scheduling information for reference by later users. Fine-grained scheduling is the scheduling of acceleration tasks of the same type.

[0085] like Figure 4 As shown, the following uses symbols to explain the multi-acceleration task scheduling system. According to the order of task submission timestamps, the acceleration tasks to be scheduled are represented as a set T = {task1, task2, ..., task n Each acceleration task contains relevant information of the task, among which the key information is the type of acceleration unit type∈{type1,type2,…,type m}. The type of the accelerator unit can be represented as accelerator∈{accelerator1,accelerator2,…,accelerator m}, where acceleration unit type type1 corresponds to acceleration unit accelerator1, and so on.

[0086] Before an acceleration task is added to the task pool, the task management algorithm first obtains the local memory usage characteristics of the acceleration task based on the historical scheduling information of this type of task, and then pre-allocates the local memory for the task; after obtaining the local memory pre-allocation information, the execution state of the acceleration task is set to "pending scheduling" and the task is added to the task pool. Figure 4 As shown, the status of tasks in the task pool includes "pending", "executing" and "suspended", which respectively correspond to the three states of the acceleration task just entering the task pool and not being scheduled to the corresponding acceleration unit task queue, the acceleration task being executed on the corresponding acceleration unit, and the acceleration task being temporarily suspended due to multiplexing with the same type of acceleration tasks on the same acceleration unit.

[0087] Task management step 202: performing polling management on the acceleration tasks in the task pool, and executing different task management strategies for each acceleration task in the to-be-scheduled, executing, and suspended states;

[0088] In a specific embodiment of the present invention, the task management step 202 further includes:

[0089] If the acceleration task is in the waiting-to-be-scheduled state, the task is assigned to different acceleration unit task queues according to the corresponding acceleration unit type, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in the waiting-to-be-scheduled state;

[0090] If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm;

[0091] If the task is in the suspended state, determine whether the execution conditions are met. If the execution conditions are met, execute the acceleration task and set the state to executing; otherwise, keep the acceleration task state as suspended.

[0092] like Figure 5As shown, the task pool management algorithm manages the tasks in the task pool. The specific management process is to poll the tasks in the task pool. If the task is in the "pending scheduling" state, the task is assigned to different acceleration unit task queues according to the acceleration unit type of the acceleration task, and the acceleration unit task queue performs more fine-grained management on the scheduling of the task. If the task is in the "executing" state, check whether the task has been completed. If the task is completed, the historical scheduling information of the same type of acceleration tasks is updated in the task management algorithm based on the task execution information such as local memory usage, task execution time, remote memory usage, etc.; if the task is not completed, determine whether the task needs to be suspended according to the scheduling algorithm (user-defined, the default is the time slice round-robin scheduling algorithm, that is, multiple tasks are time-division multiplexed on the same acceleration unit according to the time slice information); if the task is in the suspended state, check the task suspension time. When the task suspension time exceeds a certain threshold, it is necessary to check whether the corresponding acceleration unit task queue has deadlock, livelock or other task execution blocking states.

[0093] User-defined scheduling algorithms include:

[0094] 1) Time slice round-robin scheduling algorithm: set a time slice size, where the time slice size can also be customized by the user. Multiple tasks are timed during execution. If the execution time of a task reaches the set time slice size during execution, the execution of the task is suspended, the progress information of the task execution (or the task site) is retained, and the next task is switched to execute. It is similar to the execution in turns. Each task only uses the accelerator for a fixed time each time; 2) First-come-first-served scheduling algorithm: different tasks are executed in sequence according to the time when the request arrives. After each task is fully executed, it switches to the next task for execution; 3) Priority-based scheduling algorithm: set a task priority evaluation index. Multiple task requests are sorted according to the priority index, and then executed in sequence according to the sorting results. After each task is executed, it switches to the execution of the next task. Similar to the more general type of first-come-first-served scheduling algorithm, the first-come-first-served scheduling algorithm is equivalent to the task arrival time as the priority evaluation index. Different scheduling algorithms may affect the performance of multi-acceleration task execution. It is necessary to select a suitable scheduling algorithm for multiple acceleration tasks according to the actual scenario and user needs, so here it is left to the user to make a customizable design according to actual needs.

[0095] Memory management step 203: Based on the sufficiency of local memory resources and the access volume of remote memory resources, a memory balancing strategy is adopted to perform centralized memory scheduling management for acceleration tasks between local memory resources and remote memory resources.

[0096] In a specific embodiment of the present invention, the memory management step 203 further includes:

[0097] For acceleration tasks in the waiting-to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration tasks; otherwise, the current acceleration tasks are set to the suspended state;

[0098] For acceleration tasks in the execution state, if the storage request is determined to be an application for local memory resources, local memory resources will be allocated first if local memory resources are sufficient. Otherwise, the corresponding remote storage resources will be allocated to the storage request. If the storage request is determined to be an access to remote memory resources, if the remote memory access frequency exceeds the set threshold, the page will be marked as a hot page and replaced with a cold page in the local memory resource.

[0099] For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

[0100] like Figure 6 As shown in the figure, the memory management algorithm performs centralized memory management on all acceleration tasks and local and remote memory resources to balance the usage of local and remote memory and avoid frequent local and remote memory swapping, thereby reducing the memory access delay and task switching cost on the acceleration task side. For acceleration tasks in the "pending scheduling" state, memory management allocates reasonable initial local memory resources to the acceleration tasks to avoid frequent memory resource applications. When local memory resources are sufficient, memory is allocated to the acceleration tasks according to the output of the task management algorithm. If local resources are insufficient, it means that there are many current acceleration tasks and the current acceleration tasks need to wait in the task queue. For acceleration tasks in the "executing" state, the goal of memory management is to improve the utilization efficiency of local memory resources and reduce the frequency of remote memory access. Process the request for memory resources submitted by the acceleration tasks in the "executing" state, and give priority to allocating local memory resources when local memory resources are sufficient. Track the access to remote memory resources. If the remote memory access frequency of some pages exceeds the set threshold, the pages are marked as "hot pages". The less frequently accessed pages in the local memory resources need to be swapped with the "hot pages" to ensure that the more frequently accessed pages always exist in the local memory resources. Monitor memory access and proactively evict "cold pages" that have not been accessed for a long time to remote memory resources. For acceleration tasks in the "suspended" state, the goal of memory management is to reduce the number of remote memory accesses when the acceleration task in the "suspended" state switches to the acceleration task in the "executing" state. In order to help the acceleration task in the "suspended" state quickly restore the runtime memory scene when switching to the task in the "executing" state, and to efficiently utilize local memory resources, the local memory resources occupied by the acceleration task in the "suspended" state are managed by the "gradual eviction" method, and the local memory resources are gradually evicted according to the suspension time.

[0101] Among them, Figure 5 As shown, "update historical scheduling information of acceleration tasks of the same type" means that each accelerator scheduling queue will maintain the historical scheduling information of the acceleration tasks, which will include the memory usage records of the acceleration tasks, and estimate the memory usage of new acceleration tasks based on the historical memory usage values. In a specific embodiment of the present invention, it is based on the observation that tasks of the same type in the data center often have similar memory usage.

[0102] Embodiment 3

[0103] like Figure 7 As shown, the embodiment of the present application provides a multi-acceleration task scheduling system for a DPU architecture, adopting the multi-acceleration task scheduling method for a DPU architecture as described above, the DPU architecture includes multiple acceleration units, and the system includes:

[0104] The acceleration task set generation module 301 generates a set of acceleration tasks to be scheduled in the order of task submission timestamps, and constructs a task pool consisting of acceleration tasks in various states, wherein each acceleration task contains information about the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, executing, and suspended;

[0105] Task management module 302: performs polling management on the acceleration tasks in the task pool, and executes different task management strategies for the acceleration tasks in the to-be-scheduled, executing, and suspended states;

[0106] Memory management module 303: Based on the availability of local memory resources and the access to remote memory resources, a memory balancing strategy is adopted to centrally manage the memory of all acceleration tasks in local memory resources or remote memory resources.

[0107] In a specific embodiment of the present invention, the pending scheduling state of the above-mentioned acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task and the same type of acceleration tasks are suspended on the same acceleration unit due to task multiplexing.

[0108] In a specific embodiment of the present invention, the task management module 302 further includes:

[0109] Poll the acceleration tasks in the task pool. If the task is in the waiting-to-be-scheduled state, the task is assigned to different acceleration unit task queues according to the acceleration unit type of the acceleration task, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in the waiting-to-be-scheduled state;

[0110] If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm;

[0111] If the task is in the suspended state, determine whether the execution conditions are met. If the execution conditions are met, execute the acceleration task and set the state to executing; otherwise, keep the acceleration task state as suspended.

[0112] In a specific embodiment of the present invention, the memory management module 303 further includes:

[0113] For acceleration tasks in the waiting-to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration tasks; otherwise, the current acceleration tasks are set to the suspended state;

[0114] For acceleration tasks in the executing state, if the storage request is determined to be an application for local memory resources, local memory resources will be allocated first if local memory resources are sufficient, otherwise, the corresponding remote storage resources will be allocated to the request;

[0115] When it is determined that the storage request is to access the remote memory resource, and the remote memory access frequency exceeds the set threshold, the page is marked as a hot page and replaced with a cold page in the local memory resource;

[0116] For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

[0117] Embodiment 4

[0118] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned multi-acceleration task scheduling method for a DPU architecture.

[0119] Embodiment 5

[0120] like Figure 1 As shown, an embodiment of the present application provides a DPU, including a memory, a dedicated acceleration unit, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the multi-acceleration task scheduling method for the DPU architecture as described above are implemented.

[0121] In summary, the present invention builds a scheduling system suitable for multiple acceleration tasks. From the perspective of hardware resource utilization, it takes into account the computing efficiency and storage efficiency of the system, and can give full play to various hardware resources in the DPU architecture used in the data center, thereby improving the energy efficiency ratio and memory load balance of the DPU chip in the data center; from the perspective of task execution, it fully utilizes the common laws of accelerated computing and memory access of tasks of the same type, and takes into account user-defined scheduling principles, with both performance and flexibility; it fully explores the parallel characteristics of different types of acceleration tasks, optimizes the memory access behavior of multiple acceleration tasks while ensuring computing efficiency, thereby alleviating the memory access bottleneck problem existing in the parallel operation of multiple acceleration tasks, and improves the performance of the parallel operation of multiple acceleration tasks.

[0122] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A multi-acceleration task scheduling method for DPU architecture, characterized in that: The DPU architecture includes multiple acceleration units, and the method includes: The step of generating an acceleration task set is as follows: generating a set of acceleration tasks to be scheduled in the order of task submission timestamps, and constructing a task pool consisting of the acceleration tasks in various states, wherein each acceleration task contains the acceleration unit type information required for the task, and the states of the acceleration tasks include: to be scheduled, being executed, and suspended; Task management step: performing polling management on the acceleration tasks in the task pool, and executing different task management strategies for each of the acceleration tasks in the to-be-scheduled, executing, and suspended states; Memory management step: Based on the sufficiency of local memory resources and the access volume of remote memory resources, a memory balancing strategy is adopted to perform centralized memory scheduling management for the acceleration task between local memory resources and remote memory resources.

2. The multi-acceleration task scheduling method for DPU architecture according to claim 1 is characterized in that: The pending state of the acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task is suspended due to multiplexing with the same type of acceleration tasks on the same acceleration unit.

3. The multi-acceleration task scheduling method for DPU architecture according to claim 1 is characterized in that: The task management step also includes: If the acceleration task is in a waiting-to-be-scheduled state, the task is allocated to different acceleration unit task queues according to the corresponding acceleration unit type, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in a waiting-to-be-scheduled state; If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm; If the task is in a suspended state, determine whether the execution condition is met. If the execution condition is met, execute the acceleration task and set the state to be executing; otherwise, keep the acceleration task state as suspended.

4. The multi-acceleration task scheduling method for DPU architecture according to claim 1 is characterized in that: The memory management step also includes: For an acceleration task in a to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration task; otherwise, the current acceleration task is set to a suspended state; For the acceleration task in the execution state, if the storage request is determined to be an application for local memory resources, local memory resources are allocated preferentially if the local memory resources are sufficient, otherwise, the corresponding remote storage resources are allocated to the storage request; when the storage request is determined to be an access to remote memory resources, if the remote memory access frequency exceeds the set threshold, the page is marked as a hot page and replaced with a cold page in the local memory resource; For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

5. A multi-acceleration task scheduling system for DPU architecture, adopting the multi-acceleration task scheduling method for DPU architecture as claimed in any one of claims 1 to 4, characterized in that: The DPU architecture includes multiple acceleration units, and the system includes: An acceleration task set generation module generates a set of acceleration tasks to be scheduled in the order of task submission timestamps, and constructs a task pool consisting of the acceleration tasks in various states, wherein each acceleration task contains information about the type of acceleration unit required for the task, and the states of the acceleration tasks include: to be scheduled, in execution, and suspended; Task management module: performs polling management on the acceleration tasks in the task pool, and executes different task management strategies for the acceleration tasks in the to-be-scheduled, executing, and suspended states; Memory management module: Based on the availability of local memory resources and the access to remote memory resources, a memory balancing strategy is adopted to centrally manage the memory of all the acceleration tasks in local memory resources or remote memory resources.

6. The multi-acceleration task scheduling system for DPU architecture according to claim 5, characterized in that: The pending state of the acceleration task indicates that the acceleration task has just entered the task pool and has not been scheduled to the corresponding acceleration unit task queue; the executing state of the acceleration task indicates that the acceleration task is being executed on the corresponding acceleration unit; the suspended state of the acceleration task indicates that the acceleration task and the same type of acceleration tasks are suspended on the same acceleration unit due to task reuse.

7. The multi-acceleration task scheduling system for DPU architecture according to claim 5, characterized in that: The task management module also includes: Poll the acceleration tasks in the task pool. If the task is in the waiting-to-be-scheduled state, the task is assigned to different acceleration unit task queues according to the acceleration unit type of the acceleration task, and it is determined whether the task execution conditions are met. If the execution conditions are met, memory is allocated according to the pre-allocated memory situation, the acceleration task is executed, and the acceleration task state is updated to being executed; otherwise, the acceleration task state is kept in the waiting-to-be-scheduled state; If the task is in the executing state, check whether the task has been completed. If the task is completed, update the historical scheduling information of the same type of acceleration tasks based on the task execution information; if the task is not completed, determine whether the task needs to be suspended based on the default scheduling algorithm or the user-defined scheduling algorithm; If the task is in a suspended state, determine whether the execution condition is met. If the execution condition is met, execute the acceleration task and set the state to be executing; otherwise, keep the acceleration task state as suspended.

8. The multi-acceleration task scheduling system for DPU architecture according to claim 5, characterized in that: The memory management module also includes: For an acceleration task in a to-be-scheduled state, if the local memory resources are sufficient, reasonable initial local memory resources are allocated to the acceleration task; otherwise, the current acceleration task is set to a suspended state; For the acceleration task in the executing state, if the storage request is to apply for local memory resources, local memory resources are allocated first if the local memory resources are sufficient, otherwise, the corresponding remote storage resources are allocated to the request; When it is determined that the storage request is to access the remote memory resource, and it is determined that the remote memory access frequency exceeds the set threshold, the page is marked as a hot page and replaced with a cold page in the local memory resource; For the acceleration task in the suspended state, a gradual expulsion management method is adopted for the local memory resources occupied by the acceleration task in the suspended state, and the local memory resources are gradually expelled according to the suspension time.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the multi-acceleration task scheduling method for DPU architecture described in any one of claims 1 to 4 are implemented.

10. A DPU comprising a memory, a dedicated acceleration unit, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the multi-acceleration task scheduling method for DPU architecture as described in any one of claims 1 to 4 are implemented.

Images