Task processing apparatus and system

Abstract

A task processing apparatus and system are provided. The apparatus includes a storage unit, an obtaining unit configured to obtain a configuration descriptor from a host and store it in the storage unit, wherein the configuration descriptor indicates a computing task from the host and includes configuration information of the computing task, a scheduling unit configured to determine a schedulable configuration descriptor from a plurality of configuration descriptors stored in the storage unit, a plurality of computing units, each computing unit including a data prefetcher, an engine controller, and a computing engine, wherein the scheduling unit assigns the schedulable configuration descriptor to an available computing unit in the plurality of computing units, in which available computing unit the data prefetcher obtains input data from the host based on the configuration information, the engine controller configures the computing engine based on the configuration information, and the computing engine, after being configured, processes the input data to produce output data.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a task processing apparatus and system. Background Technology

[0002] With the proliferation of compute-intensive applications and the explosive growth of data volume, many application areas are placing increasingly higher demands on the data processing speed of computers. Current technologies typically address the need for large-scale concurrent data processing by increasing the computing power of general-purpose computing platforms. However, the computing power of general-purpose computing platforms often grows linearly, while data processing demands grow exponentially, creating a mismatch. To address this, existing technologies have proposed using hardware accelerators to assist general-purpose computing platforms in computation. Common hardware accelerators include application-specific integrated circuits (ASICs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and digital signal processors (DSPs), among other specialized computing hardware, to improve the processing speed of specific computational tasks, thereby increasing computational efficiency.

[0003] However, existing hardware accelerators have performance bottlenecks in multi-task concurrency scenarios and cannot fully utilize the performance of all computing engines in the hardware accelerator. Summary of the Invention

[0004] One of the purposes of this application is to provide a task processing device to fully utilize the performance of the computing engine of a hardware accelerator.

[0005] According to some aspects of this application, a task processing apparatus is provided. The task processing apparatus includes: a storage unit; an acquisition unit configured to acquire configuration descriptors from a host and store them in the storage unit, wherein the configuration descriptors indicate computing tasks from the host and include configuration information of the computing tasks; a scheduling unit configured to determine schedulable configuration descriptors from a plurality of configuration descriptors stored in the storage unit; and a plurality of computing units, each computing unit including a data prefetcher, an engine controller, and a computing engine, wherein the scheduling unit is further configured to allocate the schedulable configuration descriptors to available computing units among the plurality of computing units, such that the data prefetcher of an available computing unit acquires input data from the host based on the configuration information in the schedulable configuration descriptors, the engine controller of an available computing unit configures the computing engine of an available computing unit based on the configuration information in the schedulable configuration descriptors, and the computing engine of an available computing unit processes the input data after configuration to generate output data.

[0006] According to other aspects of this application, a task processing system is provided. The task processing system includes a host and at least one task processing device coupled to the host; the host is configured to generate a configuration descriptor indicating a computing task of the host and including configuration information of the computing task; and the task processing device includes: a storage unit; an acquisition unit configured to acquire the configuration descriptor from the host and store it in the storage unit; a scheduling unit configured to determine a schedulable configuration descriptor from a plurality of configuration descriptors stored in the storage unit; and a plurality of computing units, each computing unit including a data prefetcher, an engine controller, and a computing engine, wherein the scheduling unit is further configured to allocate the schedulable configuration descriptor to an available computing unit among the plurality of computing units, such that the data prefetcher of the available computing unit acquires input data from the host based on the configuration information in the schedulable configuration descriptor, the engine controller of the available computing unit configures the computing engine of the available computing unit based on the configuration information in the schedulable configuration descriptor, and the computing engine of the available computing unit processes the input data after configuration to generate output data.

[0007] In the task processing device of this application, a single computing task is divided into different stages and processed by different processing units, realizing the separation of control information and data information of computing tasks. Through pipelined design, the parallel execution of multiple computing engines in the task processing device is realized, thereby giving full play to the performance of these computing engines.

[0008] The above is an overview of this application, and there may be simplifications, generalizations, and omissions of details. Therefore, those skilled in the art should recognize that this section is merely illustrative and not intended to limit the scope of this application in any way. This overview section is neither intended to identify the key or essential features of the claimed subject matter nor to serve as an aid in determining the scope of the claimed subject matter. Attached Figure Description

[0009] The above and other features of this application will become more fully clear through the following description and appended claims, in conjunction with the accompanying drawings. It is understood that these drawings depict only a few embodiments of the application and should not be construed as limiting the scope of the application. The application will be described more clearly and in more detail through the use of the drawings.

[0010] Figure 1 A schematic diagram of the structure of a task processing system according to an embodiment of this application is shown.

[0011] Figure 2 A logical block diagram of a host according to an embodiment of this application is shown.

[0012] Figure 3 A schematic diagram of the structure of a task processing apparatus according to an embodiment of this application is shown.

[0013] Figure 4 A schematic diagram of the structure of a configuration descriptor according to an embodiment of this application is shown.

[0014] Figure 5 A schematic diagram of the structure of a scheduling information table according to an embodiment of this application is shown.

[0015] Figure 6 A schematic diagram of the structure of a dependency sequence list according to an embodiment of this application is shown.

[0016] Figure 7 A schematic diagram of the structure of a dependency task table according to an embodiment of this application is shown. Detailed Implementation

[0017] In the following detailed description, reference is made to the accompanying drawings, which form a part thereof. In the drawings, similar symbols generally denote similar components unless the context otherwise requires. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments and variations may be employed without departing from the spirit or scope of the subject matter of this application. It will be understood that various different configurations, substitutions, combinations, and designs can be made to the various aspects of the general description and illustrated in the drawings of this application, all of which explicitly form part of the subject matter of this application.

[0018] In existing solutions that use hardware accelerators to assist general-purpose computing platforms, a "task manager" is typically used to allocate computing tasks from the host machine to the computing engines of the hardware accelerator. However, with the continuous increase in the number of computing engines on hardware accelerators and the number of computing tasks on the host machine, the task manager has gradually become a bottleneck restricting system performance. Specifically, the task manager needs to dynamically analyze each computing task and allocate it to the appropriate computing engine. When there are many computing tasks, this leads to excessive load on the task manager. In addition, all control information and data information for computing tasks need to be processed by the task manager or allocated to the corresponding computing engines through the task manager. This centralized scheduling mode further exacerbates the load on the task manager and limits system performance.

[0019] Based on the above research on the prior art, the inventors of this application have designed a task processing device. The task processing device of this application includes an acquisition unit, a scheduling unit, a storage unit, and multiple computing units. The acquisition unit acquires configuration descriptors from the host and stores them in the storage unit. The scheduling unit determines schedulable configuration descriptors from the multiple configuration descriptors stored in the storage unit. Each computing unit in the multiple computing units of this application includes a data prefetcher, an engine controller, and a computing engine. The scheduling unit allocates schedulable configuration descriptors to available computing units among the multiple computing units. In these available computing units, the data prefetcher can acquire input data from the host based on the configuration information in the configuration descriptor; the engine controller can configure the computing engine based on the configuration information in the configuration descriptor; and the computing engine, after being configured, can process the input data to generate output data. It can be seen that the task processing device of this application differs from the task manager used in the prior art. In the task processing device of this application, a single computing task is divided into different stages and processed by different processing units, realizing the separation of control information and data information of computing tasks, and realizing the parallel execution of multiple computing engines in the task processing device through pipelined design.

[0020] refer to Figure 1 The diagram shows a schematic of the structure of a task processing system 10 according to an embodiment of the present application. The task processing system 10 includes a host 100 and a task processing device 200.

[0021] Specifically, host 100 may include processor 110, memory 120, and input / output (I / O) interface 130. Processor 110 may include one or more cores capable of executing instructions. These cores may be homogeneous and / or heterogeneous and support one or more instruction sets (e.g., x86 instruction set, MIPS instruction set, ARM instruction set, etc.). Processor 110 may be a central processing unit (CPU), or one or more of a graphics processing unit (GPU), field-programmable gate array (FPGA), accelerated processing unit (APU), or other types of processor. Memory 120 may include volatile memory and / or non-volatile memory. Memory 120 may store operating system (OS), virtual machine monitor (VMM) code, user code, and / or user data. I / O interface 130 may be any interface that allows task processing device 200 to communicate with processor 110 and / or memory 120. For example, I / O interface 130 may include a PCIe interface and a QDMA (Queue Direct Memory Access) controller. PCIe is a high-speed serial computer expansion bus standard; QDMA is a queued direct memory access technology that enables direct data transfer between memory and peripherals without relying on a processor. Those skilled in the art will understand that in other embodiments, other types of communication methods, such as PCI interfaces or DMA controllers, may be used between host 100 and task processing device 200, which will not be listed here.

[0022] The task processing device 200 is coupled to the host 100 via I / O interface 130 to interact with the host 100 for control and data information. In some embodiments, the task processing device 200 may include a storage unit 210, a fetching unit 220, a scheduling unit 230, and a plurality of computing units, for example, N+1 computing units 240-0 to 240-N, where N is an integer greater than or equal to 1. The fetching unit 220 is configured to fetch configuration descriptors from the host 100 and store them in the storage unit 210, wherein the configuration descriptors may describe computing tasks from the host 100 and include configuration information of the computing tasks. The scheduling unit 230 is configured to determine schedulable configuration descriptors from the plurality of configuration descriptors stored in the storage unit 210. Each of the plurality of computing units 240-0 to 240-N includes a data prefetcher 242, an engine controller 244, and a computing engine 246. The computing engine 246 may include hardware, software, firmware, or a combination thereof with specific computing functions. For example, the computing engine 246 can be implemented as an ASIC circuit or as an FPGA circuit. After determining the schedulable configuration descriptor, the scheduling unit 230 assigns the schedulable configuration descriptor to an available computing unit among the multiple computing units 240-0 to 240-N. In this available computing unit, the data prefetcher 242 obtains input data from the host 100 based on the configuration information in the configuration descriptor, the engine controller 244 configures the computing engine 246 based on the configuration information in the configuration descriptor, and the computing engine 246 processes the input data to generate output data after being configured.

[0023] refer to Figure 1 and Figure 2 In some embodiments, host 100 may be a computer system that supports virtualization technology. For example, host 100 may run virtual machine monitor code stored in memory 120 to create, manage, and control multiple virtual machines (VMs), such as... Figure 2 The system contains J+1 virtual machines, VM-0 to VM-J, where J is an integer greater than or equal to 1. The Virtual Machine Monitor manages the operation of these virtual machines VM-0 to VM-J by allocating necessary compute, storage, and input / output resources. These virtual machines VM-0 to VM-J can run on the same host or be distributed across multiple interconnected hosts and managed centrally by the Virtual Machine Monitor through a centralized interface provided by the Virtual Machine Monitor.

[0024] These virtual machines, VM-0 through VM-J, may be assigned to different remote and / or local users on host 100, thereby generating different computing tasks. In some embodiments, the interaction between the virtual machines and the task processing device 200 can be implemented through a work queue (WQ). A work queue can be a data structure allocated for the virtual machines in the memory 120 of host 100, and is an area for software and hardware interaction. For example, refer to... Figure 2 Each virtual machine, VM-0 through VM-J, has a corresponding queue pair (QP) QP-0 through QP-J. Each QP sequentially stores the computational tasks assigned by the corresponding virtual machine to the task processing unit 200. Each QP is independent of the others; each QP can be considered a resource exclusively for its corresponding virtual machine. (Reference) Figure 2 Each QP can include a Send Queue (SQ) and a Receive Queue (RQ). The SQ and RQ can also be referred to as the Available Queue and the Used Queue, respectively. The SQ is used to receive computation tasks from the virtual machine; the task processing unit 200 retrieves the computation tasks from the SQ and performs the computations. After the computation is completed, the task processing unit 200 can store the computation results in the RQ; then the virtual machine retrieves the computation results from the RQ.

[0025] From the perspective of the virtual machine on host 100, an exemplary processing flow for a computing task can be described as follows. First, the virtual machine prepares corresponding buffers in memory 120 for the input and output data of the computing task. Next, the virtual machine generates a configuration descriptor describing the computing task and writes it into the SQ of the corresponding QP. After writing the configuration descriptor to the SQ, the virtual machine updates the write pointer of the SQ. When the task processing device 200 detects an update to the write pointer of the SQ, it can determine that there is a new computing task in the SQ. The task processing device 200 obtains the configuration descriptor of the computing task from the SQ, performs calculations based on the configuration descriptor, and stores the information of the calculation result in the RQ. This information of the calculation result can be described using a result descriptor. The virtual machine can traverse the RQ to obtain the result descriptor of the computing task; alternatively, the virtual machine can obtain the result descriptor by using an interrupt notification indicating the completion of the computing task. In the case of using an interrupt notification, the virtual machine can read the result descriptor from the RQ when an interrupt is triggered. It should be noted that both the configuration descriptor and the result descriptor can include the identification information (ID) of the computing task. Due to differences in the priority and complexity of computational tasks, the order of configuration descriptors in SQ and result descriptors in RQ may differ. The virtual machine can determine the association between result descriptors and configuration descriptors based on ID information.

[0026] The above example illustrates the process of host 100 issuing computing tasks to task processing device 200, but this application is not limited to this. In other examples, the computing tasks issued by host 100 to task processing device 200 may also come from different applications, processes, etc.

[0027] The following combination Figure 3 The task processing apparatus 200 of this application will be further described. Figure 3 A logical block diagram of the task processing device 200 in an embodiment of this application is shown, as well as its interaction with the I / O interface 130 of the host 100.

[0028] As an example, the task processing device 200 can be implemented as a hardware accelerator coupled to the host 100, such as an expansion card or accelerator card. This expansion card or accelerator card can be deployed in the chassis of the host 100 and interconnected with the host 100 via I / O interface 130. The host 100 and the task processing device 200 can form a master-slave architecture, where the host 100 distributes computing tasks to the task processing device 200 via I / O interface 130, and the task processing device 200 is responsible for processing the computing tasks and returning the processing results to the host 100. In some embodiments, depending on the application scenario or computing requirements, multiple identical or different task processing devices 200 can be connected to a single host 100, enabling it to process multiple computing tasks in parallel to further improve task execution efficiency.

[0029] refer to Figure 1 and Figure 3 The task processing device 200 includes an acquisition unit 220. The acquisition unit 220 can be implemented using firmware and / or hardware according to design requirements. The acquisition unit 220 is configured to acquire a configuration descriptor from the SQ in the memory 120 of the host 100 via the I / O interface 130 and store it in the local storage unit 210. It should be noted that... Figure 3 The overall structure of storage unit 210 in task processing device 200 is not shown; instead, the descriptor buffer 212, scheduling information table (Slot_table) 214, pending request table (Pend_table) 216, dependency sequence table (Dep_seq_table) 215, and dependency task table (Dep_task_table) 217 ​​included in storage unit 210 are shown independently. Storage unit 210 can be implemented as volatile memory and / or non-volatile memory, so that the above-mentioned areas and lists can be stored in volatile memory and / or non-volatile memory. Although in Figure 3The diagram shows a portion of the storage areas and a list of storage units 210, but it is understood that storage units 210 may also include areas for storing other information. For example, storage units 210 may also include a local buffer maintained for each QP of host 100 to cache data from the respective QP.

[0030] Specifically, the acquisition unit 220 can store the configuration descriptors acquired from the host 100 in the descriptor buffer 212. The descriptor buffer 212 can be evenly divided into a fixed number of slots (e.g., 16 slots), each slot can be used to store one configuration descriptor. As an example, the descriptor buffer 212 can be implemented using SRAM. The configuration descriptor indicates a computing task from the host 100 and includes the configuration information of that computing task. For example, the configuration information in the configuration descriptor may include the algorithm of the computing task, the operating mode, information about the input buffers (such as address and size), and information about the output buffers.

[0031] refer to Figure 4 This shows information about a portion of the configuration descriptor fields in an example of this application. For example... Figure 4As shown, this configuration descriptor comprises 56 bytes. Byte 1 is the "Type of Command" field, which indicates the type of computational command involved in the computation task. Taking a cryptographic computation as an example, the "Type of Command" field can indicate the specific encryption algorithm type, such as a hash algorithm (e.g., Hash-based Message Authentication Code (HMAC) or Secure Hash Algorithm (SHA)), a symmetric encryption algorithm (e.g., Advanced Encryption Standard (AES)), or commercial cryptographic algorithms issued by the State Cryptography Administration of China (e.g., SM1, SM2, SM3, SM4). Byte2 and Byte3 are the "Algorithm Mode1" and "Algorithm Mode2" fields, respectively, used to indicate the operation mode of the algorithm. For example, "Algorithm Mode1" can represent the operation mode of an encryption algorithm, including but not limited to ECB (Electronic Codebook Mode), CBC (Cipher Block Chaining Mode), OFB (Output Feedback Mode), and CFB (Cipher Feedback Mode). "Algorithm Mode2" can represent the operation mode of a hash algorithm, including but not limited to SHA-256 (Secure Hash Algorithm 256-bit), SHA-384, and SHA-512. Byte5 is the "Dep_info" field, which can include dependency information of the current computation task. For example, whether the current computation task is a subtask in the task queue, and if so, its position in the task queue. The aforementioned task queue refers to a queue in which the execution of multiple subtasks is dependent, thus requiring execution in a preset order. For example, following this preset order, the execution of a subsequent subtask depends on the execution result of the preceding subtask; therefore, all subtasks in the task queue typically need to be assigned to the same computing unit for processing. In some examples, the "Dep_info" field may include a number assigned to the subtask according to a preset order to indicate its position in the task queue. Bytes 8-15 are the "TASKID" field, used to represent the identification information of the current computing task. For example, after the computing task is completed, the virtual machine can use this identification information to determine the association between the result descriptor and the configuration descriptor.Bytes 16-23 are the "Input_Address" field, used to indicate the address of the buffer (also called the "input buffer") prepared in memory 120 by the virtual machine of host 100 for the input data of the computing task. Bytes 24-27 are the "Input_Size" field, used to indicate the size of the buffer prepared in memory 120 by the virtual machine of host 100 for the input data of the computing task. Bytes 28-31 are the "Output_Size" field, used to indicate the size of the buffer (also called the "output buffer") prepared in memory 120 by the virtual machine of host 100 for the output data of the computing task. Bytes 32-38 are the "Output_Address" field, used to indicate the address of the buffer prepared in memory 120 by the virtual machine of host 100 for the output data of the computing task.

[0032] The above combination Figure 4 The configuration descriptor of one embodiment of this application has been described in detail. However, the above detailed description is only an example and is not a limitation on the technical solution of this application. Those skilled in the art can design other different types of configuration descriptors in combination with specific application scenarios and requirements.

[0033] In some embodiments, after obtaining a configuration descriptor from the host 100, the acquisition unit 220 stores the configuration descriptor in the descriptor buffer 212. Simultaneously, the acquisition unit 220 also extracts scheduling information related to the scheduling of computation tasks from the configuration descriptor and stores it in the scheduling information table 214. For example, the scheduling information table 214 includes multiple slot items, each slot item storing the scheduling information extracted from a configuration descriptor in the descriptor buffer 212. Thus, there is a one-to-one correspondence between the configuration descriptors stored in the descriptor buffer 212 and the scheduling information stored in the scheduling information table 214. The scheduling information table 214 can be used to manage the acquisition and allocation of configuration descriptors. As an example, the scheduling information table 214 may include 16 slot items, corresponding to the 16 slots in the descriptor buffer 212. Each slot item may include scheduling information related to the scheduling of computation tasks extracted from the configuration descriptor stored in the corresponding slot. Therefore, the scheduling unit 230 can manage the acquisition and scheduling of configuration descriptors based on the scheduling information in the scheduling information table 214.

[0034] refer to Figure 5 This illustrates information about the fields of a sample scheduling information table 214 in this application. For example... Figure 5 As shown, the scheduling information table 214 is divided into 16 slots, and the scheduling information stored in each slot may include:

[0035] - The “Start Address” field indicates the starting address of the configuration descriptor corresponding to this slot in the descriptor buffer 212;

[0036] - The "Pend" field indicates that a configuration descriptor read request has been issued but no response has been received. This field is cleared once a response is received.

[0037] - The "Valid" field indicates whether the current slot item corresponds to a valid configuration descriptor. For example, when

[0038] When the “Valid” field is 0, it indicates that there is no corresponding valid configuration descriptor for the current slot item; when the “Valid” field is 1, it indicates that there is a valid configuration descriptor for the current slot item that is ready for scheduling.

[0039] - The "QP" field indicates which QP the configuration descriptor corresponding to the current slot item comes from on host 100;

[0040] - The "Fmt" field indicates the format of the configuration descriptor corresponding to the current slot item;

[0041] - The "Type" field indicates the type of the configuration descriptor corresponding to the current slot item;

[0042] - The "Dep ID" field, for subtasks in the task queue, indicates the ID of the dependency buffer used to store the output data of that subtask; and

[0043] - The "Dep-Seq" field indicates the number of the subtask in the task queue.

[0044] In some embodiments, the acquisition unit 220 further maintains a dependency sequence table 215 in the memory 210 for a task queue consisting of multiple subtasks with dependencies, for recording the number of the first unprocessed subtask in the task queue in a preset order. (See reference) Figure 6The diagram illustrates a dependency sequence table 215 of this application. Taking host 100 as an example, which includes 16 QPs, the dependency sequence table 215 maintains multiple (e.g., 4) task queues for each QP of host 100, namely, next_seq[0] to next_seq[3]. The dependency sequence table 215 records the number of the first unprocessed subtask in each task queue in a preset order, which is the number of the next subtask to be processed. The scheduling unit 230 can determine the number of the next subtask to be processed in each task queue by accessing the dependency sequence table 215. Although each subtask in the task queue has a preset execution order, the configuration descriptors received by the acquisition unit 220 may be out of order. When the acquisition unit 220 receives a configuration descriptor corresponding to the next subtask to be processed in the dependency sequence table 215, the scheduling unit 230 can process the next subtask; however, when the number of the subtask corresponding to the configuration descriptor received by the acquisition unit 220 does not correspond to the next subtask to be processed, the scheduling unit 230 needs to wait temporarily. In other words, for a task queue that includes multiple subtasks with dependencies, the scheduling unit 230 can reorder the out-of-order configuration descriptors received by the descriptor acquisition unit 220 based on the dependency sequence list 215 so that each subtask can be executed in a preset order.

[0045] The above combination Figure 5 and Figure 6 The scheduling information table 214 and dependency sequence table 215 of the embodiments of this application have been described in detail. However, the above detailed description is only an example and not a limitation on the technical solution of this application. Those skilled in the art can design different types of scheduling information tables and dependency sequence tables in combination with specific application scenarios and needs.

[0046] In some embodiments, the process by which the acquisition unit 220 acquires configuration descriptors to be processed from the host 100 may include the following steps. First, the acquisition unit 220 determines whether there is available space in the descriptor buffer 212 for storing configuration descriptors to be processed. For example, the acquisition unit 220 can confirm whether there is available space in the descriptor buffer 212 by traversing the "Valid" field of each slot entry in the scheduling information table 214. When the "Valid" field of a slot entry indicates that there is no corresponding valid configuration descriptor, it can be determined that there is still available space in the descriptor buffer 212. Conversely, when the "Valid" fields of all slot entries indicate that the current slot entry corresponds to a valid configuration descriptor that is ready to be scheduled, it can be determined that there is no available space in the descriptor buffer 212. When the acquisition unit 220 determines that there is no available space in the descriptor buffer 212, it can repeatedly traverse the scheduling information table 214 until available space appears in the descriptor buffer 212. When the acquisition unit 220 determines that there is available space in the descriptor buffer 212, it can query whether there is a configuration descriptor to be processed in the SQ of each QP on the host 100 side. For example, the acquisition unit 220 can compare the write pointer and read pointer of the SQ to determine whether there are any pending configuration descriptors in the SQ. When the write pointer and read pointer are the same, it is determined that there are no pending configuration descriptors; when the write pointer and read pointer are different, it is determined that there are pending configuration descriptors. In some embodiments, pending configuration descriptors may exist simultaneously in the SQs of multiple QPs of the host 100. The acquisition unit 220 can select a QP from the multiple QPs for service according to the Quality of Service (QoS) control policy of each QP. For example, a QP with a high priority can be selected to process the pending configuration descriptors therein first. Then, the acquisition unit 220 sends a configuration descriptor read request to the QP to obtain the pending configuration descriptors from the QP. In some embodiments, such as Figure 3 As shown, the acquisition unit 220 also maintains a pending request table 216 in the memory 210, which is used to track information related to configuration descriptor read requests that have not yet been completed.

[0047] In some embodiments, the acquisition unit 220 is further configured to verify the received configuration descriptors to be processed and store the verified configuration descriptors to be processed in the descriptor buffer 212. Specifically, the acquisition unit 220 may determine whether the received configuration descriptor to be processed is consistent with the read request it issued. If there is an inconsistency (e.g., due to a system error), the acquisition unit 220 determines that the configuration descriptor to be processed is not accepted. The acquisition unit 220 may also determine whether the received configuration descriptor to be processed will cause a deadlock between multiple computational tasks with dependencies (such as multiple subtasks in a task queue). If so, the acquisition unit 220 also determines that the configuration descriptor to be processed is not accepted. When it is determined that the configuration descriptor to be processed is not accepted, the acquisition unit 220 may mark the configuration descriptor to be processed as invalid in the scheduling information table 214.

[0048] In some embodiments, storage unit 210 further includes a local buffer set up for each QP in host 100. When the configuration descriptor to be processed received by acquisition unit 220 from the host corresponds to a subtask to be processed in a new task queue, and there is currently no available computing unit to execute the new task queue, acquisition unit 220 can store the configuration descriptor to be processed in the local buffer maintained in storage unit 210 for the corresponding QP, without recording the information of the configuration descriptor to be processed in scheduling information table 214. Acquisition unit 220 can then retrieve the configuration descriptor to be processed from the local buffer for processing after an available computing unit becomes available.

[0049] In some embodiments, after the received configuration descriptor to be processed is verified, the acquisition unit 220 can extract scheduling information from the configuration descriptor and store the extracted scheduling information in the scheduling information table 214. In some embodiments, the acquisition unit 220 may also record attribute information of the configuration descriptor in the scheduling information table 214, such as the Process Address Space ID (PASID), to indicate the virtual address allocated by the host 100 to the process.

[0050] refer to Figure 3 The task processing apparatus 200 also includes a scheduling unit 230. The scheduling unit 230 can be implemented using firmware and / or hardware, depending on design requirements. The scheduling unit 230 is configured to determine schedulable configuration descriptors from a plurality of configuration descriptors in the descriptor buffer 212 based on a scheduling information table 214.

[0051] In some embodiments, the scheduling unit 230 can determine whether a schedulable configuration descriptor exists by traversing the scheduling information in each slot of the scheduling information table 214. When one of the multiple computing units 240-0 to 240-N satisfies the following conditions a) to c), the scheduling unit 230 determines that the configuration descriptor corresponding to the scheduling information of the current slot is a schedulable configuration descriptor: a) the data prefetcher 242 of the computing unit is ready to receive new computing tasks; b) the computing engine 244 of the computing unit is capable of executing the computing task indicated by the configuration descriptor corresponding to the scheduling information of the current slot; and c) when the computing task indicated by the configuration descriptor corresponding to the scheduling information of the current slot is a subtask in the task queue, other subtasks in the task queue are also assigned to the computing unit for processing. (See reference...) Figure 3 The task processing apparatus 200 also includes a demultiplexer 271. After determining that a schedulable configuration descriptor exists, the scheduling unit 230 can control the demultiplexer 271 to allocate the schedulable configuration descriptor from the descriptor buffer 212 to an available computing unit among the multiple computing units 240-0 to 240-N.

[0052] In some embodiments, the scheduling unit 230 is further configured to update the scheduling information table 214 after allocating a schedulable configuration descriptor to an available computing unit, thereby marking the slot entry corresponding to the schedulable configuration descriptor as available. This slot entry can then be reclaimed to store scheduling information for a new configuration descriptor.

[0053] In some embodiments, the scheduling unit 230 maintains a dependency task table 217 in the memory 210 for the task queue, which records information about the computing units processing the task queue and the number of the first unprocessed subtask in the task queue in a preset order. (See reference...) Figure 7The diagram illustrates a dependency task table 217 of an example of this application. Taking a host comprising 16 QPs as an example, the dependency task table 217 maintains multiple (e.g., 4) task queues for each QP of the host 100, and records information about the computing unit processing each task queue (i.e., PAgtID[0] to PAgtID[3]) and the number of the first unprocessed subtask in the task queue, which is also the number of the next subtask to be processed (i.e., Exp_seq[0] to Exp_seq[3]). For example, for the first task queue of QP-0, the value of PAgtID[0] indicates which computing unit among the multiple computing units 240-0 to 240-N is serving the task queue, more specifically, the value of PAgtID[0] indicates which computing unit's data prefetcher 242 is serving the task queue; the value of Exp_seq[0] indicates the number of the next subtask to be processed in the task queue. Accordingly, the scheduling unit 230 can determine whether the aforementioned condition c) is satisfied by accessing the dependency task table 217.

[0054] The above combination Figure 7 The dependency task table 217 of the embodiments of this application has been described in detail. However, the above detailed description is only an example and not a limitation on the technical solution of this application. Those skilled in the art can design different types of dependency task tables in combination with specific application scenarios and needs.

[0055] Continue to refer to Figure 3 The task processing apparatus 200 also includes multiple computing units 240-0 to 240-N. Each computing unit includes a data prefetcher 242, an engine controller 244, and a computing engine 246. As an example, the data prefetcher 242, the engine controller 244, and the computing engine 246 are preferably implemented in hardware. After the scheduling unit 230 allocates schedulable configuration descriptors to available computing units, the data prefetcher 242 of the available computing unit obtains input data from the host 100 based on the configuration information in the configuration descriptor, the engine controller 244 configures the computing engine 246 based on the configuration information in the configuration descriptor, and the computing engine 246 processes the input data to generate output data after being configured.

[0056] In some embodiments, host 100 may use memory virtualization technology. Data prefetcher 242 may send a request to host 100 based on the input buffer information in the configuration information above to obtain the host physical address (HPA) of the page table storing the input data. After receiving the HPA, data prefetcher 242 stores the HPA in a local translation lookaside buffer (TLB). Subsequently, data prefetcher 242 sends a data read request to host 100 based on the HPA stored in the TLB to obtain data blocks of input data, and receives the data blocks of input data from host 100. It should be noted that memory virtualization technology introduces a transition layer—the guest physical address (GPA). For host 100's VMM, it manages the actual physical memory and is responsible for allocating and managing the VM's physical memory. For the VM, it uses a virtualized GPA. Without memory virtualization technology, GPA can be regarded as the actual physical address. However, when using memory virtualization technology, the GPA cannot be directly used by the data prefetcher 242; instead, it needs to be converted into an actual HPA. Since the input data for computational tasks is typically large, it needs to be fetched from the host 100 multiple times in blocks. The data prefetcher 242 stores the HPA in its local TLB, achieving fast translation from virtual address to physical address. This avoids the repeated translation process for the virtual address of each data block, improving the efficiency of fetching input data from the host 100. Although this address translation is optional, it significantly improves data transmission efficiency, especially when the host 100 is coupled to multiple task processing devices 200. This mechanism reduces redundant translation operations and improves the overall system performance.

[0057] In some embodiments, the data prefetcher 242 can mark itself as available upon receiving the last data block of the input data, without waiting for the processing of the input data to be completed. This allows the scheduling unit 230 to allocate new computational tasks to the data prefetcher 242 as quickly as possible, thus improving the processing speed of computational tasks.

[0058] In some embodiments, the engine controller 244 is configured to use configuration information in a configuration descriptor (e.g., Figure 4The configuration descriptor shown (including the "Type of Command" field, "Algorithm Mode" field, and "Dep_info" field) configures the computing engine 246. The engine controller 244 is also used to feed input data acquired by the data prefetcher 242 to the computing engine 246, and collect the output data of the computing engine 246; and write the output data to the output buffer indicated by the configuration descriptor (e.g., ...). Figure 4 (The address indicated by the "Output_Address" field of the configuration descriptor shown). Engine controller 244 is also used to generate a result descriptor for the computation task corresponding to the schedulable configuration descriptor after processing it, and send the result descriptor to host 100, for example, by writing it into the RQ of the corresponding QP on host 100. The result descriptor can describe the result of the computation task, such as success / failure status, error code, etc.

[0059] In some embodiments, when the computation task is a subtask in a task queue, the engine controller 244, while feeding back the output data of the subtask to the host 100, also stores the output data in a dependency buffer maintained by the current computing unit, allowing other subtasks in the task queue to use it. This enables the current computing unit to directly obtain the output data of previous subtasks from the dependency buffer maintained by the engine controller 244 when processing subsequent subtasks in the task queue, without having to obtain it from the host 100, thereby improving processing efficiency.

[0060] After being configured by the engine controller 244 according to the configuration descriptor, the computing engine 246 processes input data to generate output data. The computing engine 246 is typically implemented using hardware circuitry optimized for specific computing tasks to meet diverse application requirements. For example, the computing task may be an encryption or decryption operation of input data, or a compression or decompression operation of input data. The computing engine 246 can be optimized for algorithms (such as encryption or decryption algorithms) involved in specific arithmetic instructions. The implementation of the computing engine 246 may include, but is not limited to, one or more of the following: Field Programmable Gate Array (FPGA), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), discrete devices, or transistor logic devices.

[0061] Continue to refer to Figure 3The task processing device 200 also includes a multiplexer / demultiplexer 272. The multiplexer / demultiplexer 272 can send address translation requests, data read requests, output data, and result descriptors from different computing units as described in the above embodiments to the host 100 via the communication link between the task processing device 200 and the host 100. It can also allocate HPA and input data from the host 100 to the corresponding computing units. It should be noted that, for the sake of simplicity and clarity, Figure 3 The above embodiments do not show the registers for software-hardware interaction, the logic circuits for comparing SQ write and read pointers, the logic circuits for tracking QP usage, the local buffer maintained in storage unit 210 for each QP, and other undescribed modules required to implement the task processing functions of this application, but this does not mean that these modules are absent. In some examples, the above modules or functions can be implemented together as a single virtual agent unit. The task processing device 200 may include multiple virtual agent units, each corresponding one-to-one with multiple QPs of the host 100, serving as the interface for interaction between the task processing device 200 and the host 100. In other examples, the above modules or functions may also be integrated into... Figure 3 It can be integrated into existing modules without requiring separate implementation.

[0062] Combination Figures 3 to 7As can be seen from the description of the task processing apparatus 200 of this application, the processing of computing tasks generally includes three stages: "acquiring configuration descriptors," "address translation," and "data prefetching." From the perspective of a single computing task, these three stages are executed sequentially; however, for multiple computing tasks, these three stages can be executed in parallel and interleaved. For example, in the "acquiring configuration descriptors" stage, the acquisition unit 220 sends a read request to the host and waits for a read response. While waiting to receive the configuration descriptor of a certain computing task, the communication link between the host 100 and the task processing apparatus 200 can be used for other information transmission, such as acquiring configuration descriptors of other computing tasks or acquiring input data for computing tasks. In the "address translation" stage, the data prefetcher 242 of the computing unit needs to convert the address of the input / output buffer to an HPA. During this process, the communication link between the host 100 and the task processing apparatus 200 can also be used for other information transmission. Furthermore, during the "data prefetching" phase, the data prefetcher 242 of the computing unit prefetches the input data for the newly assigned computing task without waiting for the completion of the previous computing task. This design allows the computing engine 246 to immediately switch to the new computing task after completing the previous one. While waiting for the prefetched input data, the communication link between the host 100 and the task processing device 200 can also be used for other information transmission. In other words, the task processing device 200 of this application significantly improves the throughput of data processing by efficiently utilizing the communication link between the host 100 and the task processing device 200. This makes the performance of the task processing device 200 limited only by the overall processing power of the computing engine and the bandwidth of the communication link, and not by the analysis and allocation of multiple computing tasks. Therefore, the task processing device 200 of this application can allocate computing tasks to the computing unit in advance, ensuring timely completion of data prefetching; the data flow between the host 100 and the task processing device 200 is continuous, enabling the computing engine 246 of the computing unit to execute computing tasks without interruption at full speed. In summary, in the task processing apparatus 200 of this application, the extraction of configuration descriptors and the scheduling of computation tasks are independent of the processing of computation tasks, realizing the separation of control information and data information and ensuring efficient utilization of resources; and through multi-stage processing and modular design, it supports the parallel execution of multiple computing engines, improving the throughput and overall performance of task processing.

[0063] It should be noted that the embodiments of the task processing device and system described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed in this application may be implemented through a specific interface. The indirect coupling or communication connection of devices or units may be electrical or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. In addition, the steps of the method described above may be omitted or added as needed. Furthermore, multiple steps may be executed simultaneously or sequentially. When multiple different steps are executed sequentially, their execution order may also be different in different embodiments.

[0064] Those skilled in the art can understand and implement other modifications to the disclosed embodiments by studying the specification, the disclosure, the drawings, and the appended claims. In the claims, the word "comprising" does not exclude other elements and steps, and the words "a" or "an" do not exclude a plurality. In practical applications of this application, a single part may perform the function of multiple technical features referenced in the claims. Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. A task processing device, characterized in that, The task processing device includes: Storage unit; An acquisition unit is configured to acquire a configuration descriptor from a host and store it in the storage unit, wherein the configuration descriptor indicates a computing task from the host and includes configuration information of the computing task; A scheduling unit configured to determine schedulable configuration descriptors from a plurality of configuration descriptors stored in the storage unit; and Multiple computing units, each including a data prefetcher, an engine controller, and a computing engine. The scheduling unit is further configured to allocate the schedulable configuration descriptor to available computing units among the plurality of computing units, thereby enabling the data prefetcher of the available computing unit to obtain input data from the host based on the configuration information in the schedulable configuration descriptor, the engine controller of the available computing unit to configure the computing engine of the available computing unit based on the configuration information in the schedulable configuration descriptor, and the computing engine of the available computing unit to process the input data to generate output data after being configured.

2. The task processing device according to claim 1, characterized in that, The configuration information includes the algorithm, operation mode, input buffer information, and output buffer information of the computing task.

3. The task processing device according to claim 1, characterized in that, The storage unit includes a descriptor buffer and a scheduling information table. The descriptor buffer is used to store configuration descriptors obtained from the host. The scheduling information table includes multiple slots, each slot being used to store scheduling information related to the scheduling of computing tasks extracted from a configuration descriptor in the descriptor buffer.

4. The task processing device according to claim 3, characterized in that, The acquisition unit acquires the configuration descriptor from the host, including: Determine if there is available space in the descriptor buffer; When it is determined that there is available space in the descriptor buffer, query each queue (QP) of the host to see if there are any configuration descriptors to be processed. When it is determined that the configuration descriptor to be processed exists, a read request is sent to the corresponding QP; Receive the configuration descriptor to be processed from the corresponding QP; The configuration descriptor to be processed is verified; After the configuration descriptor to be processed passes verification, the configuration descriptor to be processed is stored in the descriptor buffer; Extract scheduling information related to the scheduling of computation tasks from the configuration descriptor to be processed; and The scheduling information is stored in the scheduling information table.

5. The task processing apparatus according to claim 4, characterized in that, The acquisition unit is also configured to: When the configuration descriptor to be processed causes a deadlock between multiple computational tasks with dependencies, the configuration descriptor to be processed is marked as unvalidated in the scheduling information table.

6. The task processing apparatus according to claim 4, characterized in that, The storage unit also includes a pending request table, which is used to track read requests for the pending configuration descriptor that have not yet been completed.

7. The task processing apparatus according to claim 4, characterized in that, The computation task indicated by the configuration descriptor to be processed is a subtask to be processed in a task queue consisting of multiple subtasks with dependencies. The multiple subtasks with dependencies need to be executed in a preset order. The storage unit also includes a dependency sequence table, which is used to record the number of the first unprocessed subtask in the task queue in the preset order.

8. The task processing apparatus according to claim 7, characterized in that, The storage unit also includes a dependency task table, which is used to record information about the computing units that process the task queue and the number of the first unprocessed subtask in the task queue in the preset order.

9. The task processing apparatus according to claim 8, characterized in that, The storage unit also includes a local buffer set for each QP in the host; the fetch unit is further configured to: When the computation task indicated by the configuration descriptor to be processed is a subtask to be processed in a new task queue, and there is currently no available computation unit to execute the new task queue, the configuration descriptor to be processed is stored in the corresponding local buffer, and the information of the configuration descriptor to be processed is not recorded in the scheduling information table.

10. The task processing apparatus according to claim 3, characterized in that, The scheduling unit determines the schedulable configuration descriptors from a plurality of configuration descriptors in the storage unit, including: Traverse the scheduling information in each slot of the scheduling information table. When one of the plurality of computing units satisfies the following conditions a) to c), determine that the configuration descriptor corresponding to the scheduling information in the current slot is a schedulable configuration descriptor: a) The data prefetcher of the computing unit is ready to receive new computing tasks; b) The computing engine of the computing unit is capable of executing the computing task indicated by the configuration descriptor corresponding to the scheduling information in the current slot; and c) When the computation task indicated by the configuration descriptor corresponding to the scheduling information in the current slot item is a subtask to be processed in a task queue consisting of multiple subtasks with dependencies, the other subtasks in the task queue are also assigned to the computation unit for processing.

11. The task processing apparatus according to claim 10, characterized in that, The scheduling unit is also configured to: After assigning the schedulable configuration descriptor to the available computing unit, the scheduling information table is updated to mark the slot entry corresponding to the schedulable configuration descriptor as available.

12. The task processing apparatus according to claim 2, characterized in that, The data prefetcher of the available computing unit obtains the input data from the host based on the configuration information in the schedulable configuration descriptor, including: Based on the input buffer information in the configuration information, an address translation request is sent to the host, the address translation request being used to obtain the host physical address of the page table of the input buffer; and The host physical address is received and stored in the translation backup buffer.

13. The task processing apparatus according to claim 12, characterized in that, The data prefetcher of the available computing unit further includes obtaining the input data from the host based on the configuration information in the schedulable configuration descriptor: Send a data read request to the host using the host physical address stored in the translation backup buffer; and The data block that receives the input data from the host.

14. The task processing apparatus according to claim 13, characterized in that, The data prefetcher of the available computing unit is also configured to: When the data prefetcher receives the last data block of the input data, it marks itself as available.

15. The task processing apparatus according to claim 2, characterized in that, The engine controller of the available computing unit is also configured to: The input data is fed into the computing engine of the available computing unit; Collect the output data of the computing engines of the available computing units; as well as The output data is written to the output buffer based on the information in the configuration information.

16. The task processing apparatus according to claim 15, characterized in that, The engine controller of the available computing unit is also configured to: After the available computing unit has finished processing the computing task indicated by the schedulable configuration descriptor, a result descriptor for the computing task is generated. as well as The result descriptor is sent to the host.

17. The task processing apparatus according to claim 15, characterized in that, The engine controller of the available computing unit is also configured to: When the computation task indicated by the schedulable configuration descriptor is a subtask in a task queue consisting of multiple subtasks with dependencies, the output data of the subtask is stored in a dependency buffer in the available computing unit to allow other subtasks in the task queue to use it.

18. A task processing system, characterized in that, The task processing system includes: A host is configured to generate multiple configuration descriptors, each indicating a computing task of the host and including configuration information for that computing task; and At least one task processing device is coupled to the host, and each task processing device includes: Storage unit; An acquisition unit is configured to acquire the configuration descriptor from the host and store it in the storage unit; A scheduling unit configured to determine schedulable configuration descriptors from a plurality of configuration descriptors stored in the storage unit; and Multiple computing units, each including a data prefetcher, an engine controller, and a computing engine. The scheduling unit is further configured to allocate the schedulable configuration descriptor to available computing units among the plurality of computing units, thereby enabling the data prefetcher of the available computing unit to obtain input data from the host based on the configuration information in the schedulable configuration descriptor, the engine controller of the available computing unit to configure the computing engine of the available computing unit based on the configuration information in the schedulable configuration descriptor, and the computing engine of the available computing unit to process the input data to generate output data after being configured.

Images