A data processing task scheduling method and system for a data stream accelerator

Abstract

The application relates to a data processing task scheduling method and system for a data stream accelerator, which comprises the following steps: initializing a queue context; allocating a data packet identifier and a user virtual address based on the queue context; writing a command data packet into the user virtual address in a structured queue primitive format; generating a doorbell signal for the command data packet; converting the user virtual address into a corresponding physical address according to a queue base address and a read index in response to the doorbell signal; obtaining and analyzing the command data packet based on the physical address to determine a data processing component and an operation type; scheduling a data processing task defined by the command data packet to the data processing component for execution according to the operation type; marking the command data packet as invalid and updating the read index after the data processing component completes the task; and resetting a signal associated with the command data packet marked as invalid to indicate that the task is completed. The method significantly improves the system resource utilization and the task processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer architecture and hardware acceleration technology, specifically to a data processing task scheduling method and system for a data stream accelerator. Background Technology

[0002] Currently, with the rapid development of data centers, cloud computing, and virtualization technologies, and the increasing number of data-intensive applications, higher demands are being placed on the efficiency of processors in handling basic data operations such as memory copying, data migration, network packet forwarding, and storage deduplication. Traditional general-purpose processors often consume significant computing resources on non-core business logic when performing such operations, leading to a decline in overall system efficiency. To address this issue, the industry typically employs dedicated hardware accelerators to improve the execution efficiency of specific tasks, such as using GPUs for parallel computing, FPGAs for configurable acceleration, or dedicated ASICs for encryption and compression tasks.

[0003] However, existing hardware acceleration solutions generally have the following limitations: Most accelerators are optimized for single tasks and lack general processing capabilities across different scenarios. For example, Intel QuickAssist Technology mainly supports encryption and compression tasks; NVIDIA CUDA focuses on artificial intelligence and parallel computing; and ARM NEON is suitable for multimedia processing. These accelerators cannot simultaneously meet the needs of mixed tasks such as virtual machine migration, network protocol processing, and storage deduplication, resulting in the need to integrate multiple independent accelerators into the system, leading to hardware resource redundancy and low energy efficiency.

[0004] Existing accelerators mostly employ static task allocation strategies, lacking dynamic load balancing capabilities. The CPU needs to frequently intervene in task queue management, state synchronization, and interrupt handling, resulting in significant context switching overhead. For example, in a typical doorbell mechanism, each task completion requires triggering at least two CPU interrupts, severely limiting system throughput and real-time performance.

[0005] In virtualized environments, accelerators accessing memory require address translation between the virtual and physical memory via the IOMMU. The existing IOMMU and MMU coordination mechanism is unoptimized, resulting in high translation latency, averaging over 80ns in actual tests, far exceeding the ideal value of 30ns. This leads to significant performance degradation in scenarios such as virtual machine migration, with migration latency increasing by more than 30%.

[0006] In multi-accelerator collaborative scenarios, existing solutions such as PCIe P2P transmission still rely on the CPU to participate in data transfer, making it difficult to achieve direct and efficient communication between devices. Data transmission bandwidth utilization is typically less than 65% of the theoretical peak, failing to fully leverage the performance of high-speed interconnects.

[0007] While FPGA-based general-purpose accelerators have some reconfigurability, the time required to reconfigure the hardware logic is long and the flexibility is insufficient, making it difficult to adapt to rapidly changing business needs.

[0008] To address the aforementioned issues, while technologies such as Intel I / OAT attempt to optimize network data transmission through hardware DMA engines, they remain limited by specific protocol stacks and lack programmability and adaptability to various scenarios. Therefore, there is an urgent need for a data flow acceleration architecture that can be integrated within the processor, supports unified multi-task scheduling, possesses high programmability, low-latency virtualization support, and efficient cross-device collaboration, in order to comprehensively improve data processing efficiency and system energy efficiency in data center and cloud computing scenarios. Summary of the Invention

[0009] This invention provides a data processing task scheduling method and system for data stream accelerators, which aims to solve the problems of traditional hardware accelerators having single functions, being unable to collaborate across scenarios, and resulting in system resource redundancy and low scheduling efficiency.

[0010] To achieve the above objectives, the first aspect of the present invention provides a data processing task scheduling method for a data stream accelerator, comprising the following steps: Initialize the queue context in the packet scheduling unit; Based on the queue context, assign a packet identifier and a user virtual address in the circular buffer to the command packet to be scheduled; According to the structured queue primitive format, the command data packet containing task parameters is written to the user virtual address; A doorbell signal is generated in response to the command data packet to trigger the data packet scheduling unit; In response to the doorbell signal, the user virtual address is converted into the corresponding physical address based on the queue base address and the read index; The command data packet is obtained and parsed based on the physical address to determine the data processing component and operation type specified therein. According to the operation type, the data processing task defined by the command data packet is scheduled to be executed by the data processing component; After the data processing unit completes its task, it marks the command data packet as invalid and updates the read index. Reset the signals associated with the command data packet that has been marked as invalid to indicate that the task has been completed.

[0011] Furthermore, the data processing component is a hardware data processing component integrated in the data stream accelerator, and the component includes at least one of the following: a data comparator, a cyclic redundancy check unit, an address translation and migration unit, a data packing and unpacking unit, and a protocol parsing and offloading unit.

[0012] Furthermore, the structured queue primitive format is a fixed-size 64-byte data packet format, including a header field, a parameter field, and a pointer field to the working set.

[0013] Furthermore, the doorbell signal is triggered by the host processor by writing to the doorbell register or a shared memory location, and is used to notify the data packet scheduling unit that there is a new command data packet to be processed.

[0014] Furthermore, it also includes the step of prefetching the data to be processed into a local buffer through the DMA engine embedded in the packet scheduling unit during the execution of the task by the data processing unit.

[0015] Furthermore, in a virtualized environment, the translation of the user's virtual address to a physical address includes the following steps: In the outbound direction, the translation from virtual address to physical address is completed by the host processor's memory management unit; In the inbound direction, address translation is performed when the accelerator accesses memory through the input / output memory management unit.

[0016] Furthermore, the packet scheduling unit consists of a hardware scheduler and runtime software, with the runtime software running on a designated RISC-V processor core or a dedicated RISC-V microcore integrated within a data stream accelerator.

[0017] Furthermore, the method includes setting a barrier data packet in the circular buffer, wherein the barrier data packet is used to indicate that all command data packets preceding the barrier must be executed before subsequent data packets can be executed.

[0018] Furthermore, the step of resetting the signal associated with the marked invalid command data packet includes setting a completion flag in a shared memory location to wake up a host application waiting for the task to complete.

[0019] To achieve the above objectives, a second aspect of the present invention provides a data stream accelerator system, comprising: The host processor, a RISC-V architecture processor core, is used to execute applications and generate data processing tasks; A data stream accelerator, connected to the host processor, includes multiple hardware data processing components for performing at least one of the following operations: data comparison, verification, migration, format conversion, and protocol offloading. The packet scheduling unit, including a hardware scheduling engine and runtime software, is connected to the host processor and the data stream accelerator and is used to schedule data processing tasks. Shared memory, including a circular buffer, for storing command data packets written by the host processor; A doorbell register or shared memory location is used to receive doorbell signals sent by the host processor; The address translation unit, including the memory management unit and the input / output memory management unit, is used to perform virtual-to-physical address translation in a virtualized environment; The packet scheduling unit is configured as follows: Receive the queue context initialized by the host processor through the runtime software; Assign packet identifiers and user virtual addresses based on the queue context; In response to the doorbell signal, the user's virtual address is converted into a physical address based on the base address and the read index; Parse the command data packet at the physical address to determine the target data processing unit and operation type; The task is scheduled to be executed by the corresponding data processing component. After the task is completed, update the read index and reset the completion signal.

[0020] The beneficial effects of this invention are: Compared with existing technologies, this invention provides a data processing task scheduling method and system for data stream accelerators. By introducing a unified structured queue primitive format as a standardized description language for all data acceleration tasks, and designing a centralized, programmable hybrid scheduler to parse this format and dynamically schedule tasks, it effectively solves the problems of traditional hardware accelerators having single functions, being unable to collaborate across scenarios, and resulting in system resource redundancy and low scheduling efficiency. The technical solution involves the application encapsulating data processing tasks of different natures into command data packets according to a unified structured queue primitive format. The hybrid scheduler then uniformly parses these data packets at the hardware level and accurately assigns them to the corresponding dedicated data processing components for execution based on their embedded operation types. This solution fundamentally solves the problem because it breaks the rigid design paradigm of traditional accelerators—"one task, one set of hardware"—by collaborating with software definition and hardware execution. Without changing the hardware architecture, it flexibly supports emerging data acceleration tasks simply by extending the structured queue primitives and scheduling strategies. This achieves efficient collaborative processing of mixed workloads across multiple scenarios using a unified, programmable framework, significantly improving system resource utilization and task processing efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0022] Figure 1This is a flowchart of a data processing task scheduling method for a data stream accelerator disclosed in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of a programmable dataflow accelerator architecture disclosed in an embodiment of the present invention.

[0024] Figure 3 This is an operational flow and data structure framework diagram of an application offloading to an on-chip DSA accelerator disclosed in an embodiment of the present invention.

[0025] Figure 4 This is a DSA communication protocol framework diagram disclosed in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] According to embodiments of the present invention, it should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the following methods, in some cases the steps shown or described may be executed in a different order than that shown here.

[0028] like Figure 1 As shown, the present invention provides a data processing task scheduling method for a data stream accelerator, comprising the following steps: Step S100: Initialize the queue context in the packet scheduling unit; In some embodiments, this step is performed by specific software running on the host processor (i.e., packet scheduling unit runtime software), the core purpose of which is to establish a controlled and efficient communication channel for subsequent command submission and scheduling.

[0029] Specifically, a "queue context" refers to a set of hardware and software configuration parameters used to define and manage a task queue. Its main components include: the starting location (base address) of the circular buffer corresponding to the queue in system memory, the size of the buffer, the queue's access attributes (such as privilege level), and initial state information related to the queue (such as the initial values ​​of the read / write indexes). This initialization process is completed by calling a dedicated application programming interface (API) provided by the packet scheduling unit at runtime. The host processor executes this API to write the above configuration parameters into the packet scheduling unit's hardware registers or reserved configuration memory area, thereby completing the establishment of the queue context.

[0030] Step S200: Based on the queue context, assign a data packet identifier and a user virtual address in the circular buffer to the command data packet to be scheduled; In some embodiments, this step is performed by an application running on the host processor calling a dedicated interface function provided by the packet scheduling unit runtime. When an application generates a data processing task that needs to be offloaded to the data stream accelerator for execution, it first requests a task slot from the packet scheduling unit runtime.

[0031] The "packet identifier" is a unique sequence number dynamically generated and managed by the packet scheduling unit at runtime, used to identify the specific command packet throughout the task's lifecycle. Subsequently, based on the queue context established in step S100, specifically the base address of the circular buffer and the predefined fixed packet size (e.g., 64 bytes), the system determines the "user virtual address" corresponding to the command packet in the circular buffer by calculating the "base address, packet identifier, and packet size." This address is a memory location visible and addressable by the application, used for subsequently writing specific task instructions. This allocation mechanism ensures that each pending task obtains an ordered and independent storage location within the shared circular buffer.

[0032] Step S300: Write the command data packet containing task parameters to the user virtual address according to the structured queue primitive format; The Structured Queue Primitive Format (SQP) is a standardized, fixed-size packet format defined in this implementation. Each SQP packet, for example, can be a fixed length of 64 bytes, and its internal structure is predefined to include several key fields: a "header" field, which indicates the type of packet (such as a kernel scheduling packet, agent scheduling packet, or barrier packet) and basic control information; an "operation type and parameters" field, which specifies the specific acceleration operation to be performed (e.g., 'DSAx_compare' for memory comparison, 'DSAx_transform' for data transformation, 'DSAx_Move' for data migration, etc.) and the configuration parameters required for the operation (such as the starting address of the comparison, data length, and verification algorithm type, etc.); and one or more "pointers" fields, which point to the actual working dataset of the task in system memory (i.e., the physical memory area where the data to be processed resides).

[0033] After obtaining the specific write location (user virtual address) in the circular buffer in step S200, the application fills the opcode, parameters, and data pointer corresponding to this task into the 64-byte memory block pointed to by that address according to the predefined SQP format. For example, if the task is "to calculate the CRC check value of a memory region", the application will fill in 'DSAx_transform' (or its corresponding specific sub-encoding) in the operation type field, specify the CRC-32 algorithm in the parameter field, and fill in the starting address and length of the memory region to be checked in the pointer field. Through this unified and structured encapsulation method, diverse tasks from different application scenarios at the upper layer (such as virtual machine migration, network packet processing, and storage deduplication) are all converted into standardized "data packets" that can be consistently parsed by the hardware scheduler.

[0034] Step S400: Generate a doorbell signal for the command data packet to trigger the data packet scheduling unit; A "doorbell signal" is a software-originating, hardware-initiated trigger event. It can be implemented in two ways: first, the host processor performs a write operation to a specific, memory-mapped "doorbell register" in the packet scheduling unit hardware; second, the host processor writes a specific value to a pre-agreed "shared memory location" polled by the hardware. Regardless of the method used, this write operation itself constitutes the doorbell signal. This signal does not carry complex task information; its sole semantic meaning is to notify the packet scheduling unit that at least one new command packet (its position implicitly indicated by the queue's "write index") is ready for scheduling execution in a specific task queue (identified by the queue context).

[0035] After the application completes the command data packet in SQP format, it only needs to perform a doorbell signal generation operation once to complete the task delivery process, without waiting for hardware response or immediate state synchronization. The data packet scheduling unit hardware automatically initiates the subsequent scheduling process after detecting the doorbell signal in the background. This contrasts sharply with traditional solutions where each task completion may require CPU intervention and multiple context switches to handle interrupts and state synchronization.

[0036] Step S500: In response to the doorbell signal, the user virtual address is converted into the corresponding physical address based on the queue base address and the read index; Specifically, the packet scheduling unit first locates the queue context initialized in step S100 based on the queue identifier associated with the triggered doorbell signal. It then obtains the starting position of the queue's circular buffer in system memory, i.e., the "queue base address." Next, it obtains the queue's current "read index," a pointer maintained internally by the packet scheduling unit hardware, indicating the logical position of the next command packet to be read in the circular buffer.

[0037] The packet scheduling unit can calculate the logical offset of the command packet to be processed in memory. However, the application operates on a "user virtual address" in steps S200 and S300. In order for the hardware to directly access memory to obtain the packet content, this user virtual address must be translated into a "physical address" that the hardware can directly address.

[0038] In a preferred embodiment of the present invention, particularly in a virtualization-supporting environment, this conversion process is completed through hardware and software collaboration, forming an "IOMMU and MMU collaborative virtualization architecture." The specific path is as follows: the packet scheduling unit, acting as a "device" in the system, initiates a read operation in the "Inbound" direction (i.e., device access to memory). This access request (containing the target user virtual address) is first submitted to the input / output memory management unit. The IOMMU, based on the address translation table configured for the packet scheduling unit (or its associated dataflow accelerator), translates the user virtual address into an intermediate physical address or directly into a system physical address. This process is entirely completed by hardware, without the need for host processor intervention, thereby eliminating the latency caused by address translation triggered by CPU interrupts in traditional solutions.

[0039] Step S600: Obtain and parse the command data packet based on the physical address to determine the data processing component and operation type specified therein; The packet scheduling unit first uses its integrated direct memory access engine to read the entire command packet (e.g., a complete 64-byte content) from system memory into its local buffer or register set based on the physical address obtained in step S500. The reading process is performed via a DMA channel, eliminating the need for the host processor to handle the data. Subsequently, the scheduling engine parses the packet according to a predefined structured queue primitive format. The parsing process first identifies the packet's header fields to confirm whether it is a valid scheduling packet (e.g., a kernel scheduling packet). Next, the engine extracts the operation type field, which is an encoded instruction that explicitly specifies the acceleration operation requested by the packet, such as 'DSAx_compare' (memory comparison), 'DSAx_Move' (data migration), or 'DSAx_transform' (data transformation / verification).

[0040] After successfully decoding the operation type, the packet scheduling unit needs to determine the optimal or designated data processing unit to execute the task based on this type. In this embodiment, the data stream accelerator integrates a series of hardware modules dedicated to different basic operations, such as a data comparator, a cyclic redundancy check unit, an address translation and migration unit, a data packing and unpacking unit, and a protocol parsing and offloading unit. The packet scheduling unit maintains a mapping relationship (implemented through hardware logic or a configurable lookup table) that associates different SQP operation type codes with one or more available physical data processing units. For example, when the operation type is parsed as 'DSAx_compare', the scheduling logic maps it to a hardware comparator unit. Simultaneously, the packet scheduling unit extracts all the parameters required for the operation (such as data length and algorithm mode) and a memory pointer to the actual data to be processed from the data packet. This information, along with the operation type, constitutes a complete "task descriptor".

[0041] Step S700: According to the operation type, schedule the data processing task defined by the command data packet to the data processing component for execution; In practical implementation, the packet scheduling unit internally includes a task dispatcher. Based on the target data processing unit determined in step S600 and its current state (e.g., idle, busy, buffer full), the dispatcher sends a complete task descriptor to that unit via an on-chip network or hardware interconnect channel. The task descriptor is encapsulated into an internal transport packet, which not only contains the opcode and parameters extracted from the SQP packet but also includes scheduling information generated by the scheduling unit, such as task priority and source queue identifier.

[0042] Once the target data processing unit receives the task descriptor, its internal control logic is activated. This logic configures the internal computational pipeline or state machine based on the operation type. More importantly, it initiates a data acquisition operation based on the memory pointer carried in the task descriptor. This operation is performed through a direct memory access engine integrated into the unit itself or shared by the packet scheduling unit, thereby directly transferring the raw data to be processed from the system main memory to the unit's local input buffer without any intervention from the host processor.

[0043] Once the data and instructions are ready, the hardware data processing unit immediately begins executing its dedicated accelerated operations. For example, if the task is 'DSAx_fill' (memory fill), the unit writes the data to the target memory region according to the specified pattern; if it is 'DSAx_compare', it compares the two memory regions byte by byte and generates the result status. The entire execution process is autonomous at the hardware level, working in parallel with the host processor and other components, achieving true computational offloading and parallel processing.

[0044] Step S800: After the data processing unit completes its task, the command data packet is marked as invalid and the read index is updated; When a data processing unit (such as a data comparator) completes its assigned task (such as memory comparison), it sends a completion notification to the packet scheduling unit. This notification is either a hardware interrupt signal or the setting of a specific completion flag in a shared status register. Upon receiving this notification, the packet scheduling unit immediately performs hardware management operations.

[0045] First, the command packet corresponding to the task is marked as invalid in the circular buffer. This operation is not a physical erasure of the data, but rather an update of the packet's position status from "pending" or "in execution" to "processed / invalid" in the status bitmap maintained internally by the packet scheduling unit and associated with each slot in the circular buffer. This status information is visible to the packet scheduling unit itself, allowing the scheduler to skip processed slots in subsequent scheduling scans. However, for the host application, the contents of that slot in the circular buffer may remain unchanged until explicitly overwritten by software. Marking it invalid is essentially a logical release of the task slot from the scheduling perspective.

[0046] Secondly, the read index of the queue is updated. The read index is a pointer maintained by the packet scheduling unit hardware, indicating the logical position of the next command packet to be read (i.e., scheduled) in the circular buffer. After confirming that a packet has been processed and marked as invalid, the hardware logic of the packet scheduling unit automatically increments the read index, pointing to the next sequential slot in the circular buffer.

[0047] This step prevents outdated tasks from being incorrectly rescheduled by marking them as invalid, while updating the read index drives the task queue forward, making room and indicating the location for receiving and processing new tasks, thereby maintaining the high throughput and low latency continuous processing capability of the entire data flow acceleration system.

[0048] Step S900: Reset the signal associated with the marked invalid command data packet to indicate that the task is complete.

[0049] In step S300, when the application writes a command packet to the circular buffer, a completion signal is associated with it. This signal is essentially a specific variable (e.g., an integer) pre-allocated and initialized by the application in shared memory, or a reverse notification register mapped to the doorbell register space. Its initial state is set to "incomplete". In step S800, after the packet is marked as invalid and the read index is updated, the packet scheduling unit executes this step. The hardware scheduling logic finds the physical memory address or register address of the associated completion signal based on the identification information carried in the command packet for the completed task, and then performs a write operation. This write operation sets the completion flag in shared memory to the "complete" state.

[0050] This method utilizes a hybrid scheduler (DPSU) comprised of hardware components and runtime software, combined with standardized structured queue primitives (SQPs, such as a 64-byte fixed format), to achieve unified dynamic scheduling and offloading of various task types across AI accelerators, network interface cards (NICs), and memory management. Compared to existing technologies that require deploying independent accelerators for different functions, this solution can schedule hybrid tasks with a single DPSU, improving resource utilization several times over. Furthermore, adding new operation types only requires modifying the software interface, without hardware refactoring. Task triggering and state synchronization are driven by a doorbell signal, with hardware autonomously updating the queue index. Compared to traditional solutions (such as ARM DMA) which require two CPU interrupts for context switching per task, this solution significantly reduces CPU intervention frequency and task latency. Secondly, for scenarios such as virtual machines, a collaborative address translation closed loop is constructed, with the inbound direction managed by the IOMMU and the outbound direction completed by the CPU MMU. This optimization significantly reduces the average time for virtual-to-physical address translation; according to tests, virtual machine migration efficiency can be improved by approximately 50%. Additionally, a DMA engine is integrated within the DPSU, supporting the prefetching of subsequent data packets to the local buffer during accelerator activity. This mechanism effectively avoids frequent CPU intervention in data transfer, thereby improving the bandwidth utilization of cross-device (such as PCIe P2P) data transfer, which can approach the theoretical peak, while reducing system power consumption.

[0051] This embodiment provides a unified and efficient hardware and software infrastructure for implementing the application model. The detailed process described above from steps S100 to S900 essentially constructs a complete closed loop from application task submission to hardware-accelerated execution and completion notification. The core of this closed loop lies in transforming the complex and variable data processing requirements of upper-layer applications into a series of atomic operations that can be executed in parallel and pipelined by dedicated hardware components through a programmable packet scheduling unit (DPSU) and standardized structured queue primitives (SQP).

[0052] Based on this general scheduling framework, the DSA of this invention can flexibly adapt to various design patterns such as "data processing offloading," "storage data migration," "support for virtual machine data migration," and "data movement between peer-to-peer I / O devices." For example, when an application needs to perform deduplication in memory (corresponding to the "data processing offloading" mode), the application can generate an SQP data packet with operation type 'DSAx_compare' and point it to the memory page to be compared. The DPSU will schedule it to the integrated hardware comparator unit for execution, quickly determining whether the pages are equal, thereby efficiently supporting deduplication operations. For scenarios that require generating CRC checks (corresponding to the "storage data migration" mode), a dedicated check unit can be called through the 'DSAx_transform' operation type.

[0053] In complex scenarios involving virtual-to-physical address translation and efficient data movement, such as supporting virtual machine migration, its "IOMMU and MMU collaborative virtualization architecture" ensures efficient and transparent address translation when the accelerator accesses virtual machine memory. When the Virtual Machine Management Module (VMM) needs to identify and migrate modified pages, it can initiate a series of SQP operations such as 'DSAx_compare' (compare before and after snapshots to generate differences) and 'DSAx_Move' (migrate the difference data). These operations are dynamically scheduled by the DPSU to be executed by the corresponding hardware components, and the embedded DMA engine directly moves data between the source and target memory (or device), thereby significantly reducing CPU load and network traffic and achieving a fast and efficient migration process.

[0054] To achieve the aforementioned efficient task scheduling and execution mechanism, this embodiment proposes and implements a hierarchical programmable dataflow accelerator (DSA) design in terms of hardware architecture. The core of this design is a hardware-software co-operational hybrid scheduler—the Packet Scheduling Unit (DPSU). For example... Figure 2As shown, the DPSU, as a key scheduling hub in the System-on-a-Chip (SoC), connects upwards to the computing resource pool of the host RISC-V processor core via a standard interface, and downwards manages and connects one or more dedicated dataflow acceleration units (DSA0~m). The DPSU itself consists of a hardware engine (DPSU Hardware) and runtime software (DPSU Runtime), a hybrid design that gives the system a high degree of programmability and flexibility. The DPSURuntime runs on a DSA Agent entity, which can be either a general-purpose RISC-V processor core specified by software or a dedicated RISC-V microcore integrated within the DSA hardware, responsible for control plane functions such as accelerator initialization, resource allocation, and task supervision.

[0055] The programmable DSA architecture constructed in this embodiment can be logically divided into three closely cooperating layers. The top layer is the application layer, covering specific applications such as AI acceleration, high-speed NIC communication, CXL acceleration, and video encoding / decoding. The middle layer is the system software layer, including the operating system, DSA runtime library, scheduler, and firmware. This layer provides a bridge for securely and efficiently offloading user-level task requests to the underlying hardware through software components such as the DPSU Runtime, and manages user-level command queues based on Structured Queuing Primitives (SQP). The bottom layer is the hardware architecture layer, containing a general-purpose computing core, multiple configurable DSA acceleration components, on-chip networking, a memory subsystem, and I / O expansion interfaces for connecting external high-speed devices (such as AI acceleration cards and smart network cards with PCIe / CXL interfaces). This layered design clearly isolates applications, system software, and hardware, allowing innovation to focus within each layer while maintaining overall high-efficiency collaboration through standardized interfaces.

[0056] In terms of hardware implementation, multiple heterogeneous DSA acceleration components (such as comparators, checkers, and migration engines) are connected to the DPSU hardware scheduler via an on-chip network. The DPSU can dynamically schedule these components in a shared mode or a subset mode based on the application load. Figure 2 As shown in the diagram, this illustrates a SoC system-level abstraction view integrating a DPSU scheduling hub, multiple DSA acceleration components, a host core, and external I / O interfaces. This integration allows diverse tasks from the application layer to be parsed by the DPSU and distributed to the most suitable dedicated hardware units for parallel execution through a unified queue interface at the system software layer. Simultaneously, through high-speed I / O expansion interfaces, the on-chip DSA architecture can also efficiently interact with off-chip coprocessors (such as GPUs and FPGA accelerator cards), enabling broader task offloading and data transfer between accelerators.

[0057] like Figure 3As shown in the diagram, this illustration specifically demonstrates how an application unloads a job to the DSA accelerator using the SQP packet format, and the collaborative process between the DPSU runtime and the hardware. The application encapsulates the tasks (jobs) to be unloaded into fixed-size (e.g., 64 bytes) SQP packets with a strictly defined internal structure, including a Header field to distinguish the packet type (e.g., kernel scheduling packet, agent scheduling packet, or barrier packet), the specific operation command (Opcode), and a pointer to the working data. SQP defines a rich set of control commands, such as 'DSAx_Move' for data migration, 'DSAx_compare' for memory comparison, 'DSAx_transform' for data transformation or verification, and 'DPSU_barrier' for setting execution barriers, etc.

[0058] Packets are organized in a user-level command queue managed by the DPSU. This queue is organized using a "semi-transparent" mechanism: its software-visible parts include the queue context and a circular buffer, which applications can directly access via the base address to fill or release SQP packets; while its hardware management parts, such as the read / write index, are transparent to the software and are maintained and updated by the DPSU runtime or hardware engine.

[0059] Figure 3 This demonstrates the coordination process from application job submission to DSA hardware execution completion. In this process, the DPSU runtime is responsible for three main functions: first, receiving and managing job unloading requests from user-space applications, caching them in a circular queue; second, coordinating the entire unloading process, including transferring job descriptions and data from user space to the hardware accelerator and retrieving processing results; and third, handling concurrency synchronization, notifying the DSA agent (i.e., the RISC-V core or microcore responsible for management) of task completion status through an event signaling mechanism. The DPSU hardware engine, based on the instructions coordinated by the runtime, retrieves SQP packets from the queue, parses them, and, according to the specified job attributes (i.e., operation type), assigns specific data processing tasks to corresponding dedicated hardware components (such as comparators, check units, migration engines, etc.) for execution. When necessary, it also schedules the built-in DMA engine to efficiently move data between host memory, on-chip buffers, and external I / O devices.

[0060] Based on the aforementioned task scheduling mechanism, the DSA agent (host CPU or dedicated microcore) and the DSA packet scheduling unit (DPSU) collaborate through a well-defined and efficient communication protocol. This protocol, centered on a user-level queue sharing memory space, enables a unidirectional task flow from the host to the accelerator hardware. Its complete interaction process includes three phases: startup, activation, and completion. For example... Figure 4As shown in the diagram, this flowchart details the complete communication protocol and state transition process from the host CPU initializing the queue, submitting commands, triggering the doorbell, to the DPSU scheduling execution and returning a completion signal. The protocol outlines the complete path from the Host CPU (i.e., the RISC-V core) initiating the task, through coordination between the DSA Agent (a dedicated hardware thread or an internal DSA microkernel) and the DSU component's scheduler (DPSU), to the final execution by the DSA data processing unit and the return of the result. The arrows in the diagram clearly indicate key steps such as "creating and allocating queue context," "generating and writing command / primitive messages," "triggering the doorbell," "scheduling," and "sending a completion signal," visually demonstrating the task flow and state transitions between software and hardware.

[0061] Specifically, during the startup phase, the host CPU first calls the DPSU runtime API to initialize the target queue's context, configuring its base address, size, and other key parameters to establish a controlled environment for communication. Then, the activation phase begins: the application requests a unique packet ID through the runtime interface, and then calculates the user virtual address (userVA) of that packet in the circular buffer; next, according to the predefined Structured Queue Primitives (SQP) format, it fills the specific operation commands, parameters, and data pointers into the buffer location corresponding to that address. This process corresponds to... Figure 4 The process involves "Write command / primitive message generation" in the diagram. Finally, the application sets a task completion signal at the associated shared memory location and notifies the DPSU that a new task is ready via a "doorbell" write operation, triggering the "Doorbell" step in the diagram. The DPSU hardware continuously monitors the doorbell signal, and once triggered, it enters the scheduling phase. Based on the queue base address and the internally maintained read index (corresponding to "read index / pointer" in the diagram), with the assistance of the address management unit (such as the IOMMU), it translates the userVA into a physical address (PA), then acquires and parses the SQP data packet, and dispatches it to the corresponding DSA processing unit for execution according to its type. This is the "scheduling" and "initiation dispatch" process. The DPSU continuously schedules data packets in the queue until it encounters a barrier packet, which serves as a synchronization point. This barrier ensures that all tasks preceding it are completed before subsequent tasks are initiated; this synchronization point is indicated by "Barrier flag = 1" in the diagram.

[0062] During the completion phase, when a DSA processing unit (such as DSA[x]) completes its task, it notifies the DPSU. The DPSU then performs two hardware management operations: first, it marks the data packet corresponding to the task as invalid in its internal state; second, it updates the read index of the queue to advance the processing progress. Finally, the DPSU resets the completion signal associated with the data packet (i.e., writes a completion flag to the shared memory location, corresponding to "sending a completion signal" in the diagram), thereby waking up the waiting host application and allowing "distribution of new Head data packets" to continue execution behind the barrier. This protocol, which involves software-triggered doorbell startup, hardware autonomous scheduling and state maintenance, and finally synchronization via signals, greatly reduces the number of times the host CPU intervenes in task distribution and state synchronization, achieving efficient and low-latency data stream offloading communication.

[0063] Building upon the aforementioned general task scheduling and communication protocols, this embodiment of DSA further integrates a fast data transformation mechanism, enabling efficient native operations on memory data directly through dedicated hardware components. This mechanism allows DSA to perform advanced transformations directly in memory space, such as Cyclic Redundancy Check (CRC) generation and verification, memory region comparison, data rearrangement, and migration. For example, in a virtual machine real-time migration scenario, DSA can continuously execute the 'DSAx_compare' operation to quickly compare memory pages and generate incremental difference records. Subsequently, 'DSAx_Move' is used to migrate only the difference data, thereby significantly reducing network traffic and migration latency. In storage or network data paths, a dedicated verification unit can seamlessly execute 'DSAx_transform' to complete data integrity protection. All these transformation operations are abstracted into standard SQP primitives, uniformly scheduled by the DPSU to the corresponding hardware components for execution. The entire process from data movement and computation to result write-back is completed at the hardware level without CPU intervention, thus achieving extremely low latency and high throughput data processing in applications such as virtual machine checkpointing, memory deduplication, and high-speed network packet processing.

[0064] This solution incorporates two hardware enhancements. First, a dedicated fast DMA engine is integrated within the DPSU. This engine proactively prefetches one or more subsequent SQP packets and their associated working datasets from main memory to the local buffer of the DPSU or DSA unit while the DSA acceleration unit is active. This advance data fetching effectively hides memory access latency, ensuring a continuous data supply to the computing unit, thereby significantly improving pipeline efficiency and overall throughput. Second, complete DSA virtualization support is implemented, with its core being the collaborative address management between the IOMMU and the host CPU MMU. Figure 2 and Figure 4As shown, on the inbound path from the accelerator to memory, the IOMMU integrated in the DPSU or interconnect is responsible for transparently translating the virtual address (VA) request initiated by the device into a physical address (PA), ensuring that the accelerator can correctly and safely access virtual machine memory in the virtualization environment. On the outbound path from memory to the accelerator, the address translation during the task submission process is routinely completed by the host CPU running DSAAgent through its MMU. This bidirectional, collaborative address translation closed loop enables DSA to seamlessly and efficiently serve virtualization scenarios such as virtual machine migration and inter-container communication, virtually eliminating additional address translation overhead.

[0065] According to another aspect of the embodiments of this application, a second aspect of the present invention provides a data stream accelerator system, comprising: The host processor, a RISC-V architecture processor core, is used to execute applications and generate data processing tasks; A data stream accelerator, connected to the host processor, includes multiple hardware data processing components for performing at least one of the following operations: data comparison, verification, migration, format conversion, and protocol offloading. The packet scheduling unit, including a hardware scheduling engine and runtime software, is connected to the host processor and the data stream accelerator and is used to schedule data processing tasks. Shared memory, including a circular buffer, for storing command data packets written by the host processor; A doorbell register or shared memory location is used to receive doorbell signals sent by the host processor; The address translation unit, including the memory management unit and the input / output memory management unit, is used to perform virtual-to-physical address translation in a virtualized environment; The packet scheduling unit is configured as follows: Receive the queue context initialized by the host processor through the runtime software; Assign packet identifiers and user virtual addresses based on the queue context; In response to the doorbell signal, the user's virtual address is converted into a physical address based on the base address and the read index; Parse the command data packet at the physical address to determine the target data processing unit and operation type; The task is scheduled to be executed by the corresponding data processing component. After the task is completed, update the read index and reset the completion signal.

[0066] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0067] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0068] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0070] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing task scheduling method for a data stream accelerator, characterized in that, Includes the following steps: Initialize the queue context in the packet scheduling unit; Based on the queue context, assign a packet identifier and a user virtual address in the circular buffer to the command packet to be scheduled; According to the structured queue primitive format, the command data packet containing task parameters is written to the user virtual address; A doorbell signal is generated in response to the command data packet to trigger the data packet scheduling unit; In response to the doorbell signal, the user virtual address is converted into the corresponding physical address based on the queue base address and the read index; The command data packet is obtained and parsed based on the physical address to determine the data processing component and operation type specified therein. According to the operation type, the data processing task defined by the command data packet is scheduled to be executed by the data processing component; After the data processing unit completes its task, it marks the command data packet as invalid and updates the read index. Reset the signals associated with the command data packet that has been marked as invalid to indicate that the task has been completed.

2. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, The data processing component is a hardware data processing component integrated in the data stream accelerator. The component includes at least one of the following: a data comparator, a cyclic redundancy check unit, an address translation and migration unit, a data packing and unpacking unit, and a protocol parsing and offloading unit.

3. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, The structured queue primitive format is a fixed-size 64-byte data packet format, including a header field, a parameter field, and a pointer field to the working set.

4. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, The doorbell signal is triggered by the host processor by writing to the doorbell register or a shared memory location, and is used to notify the data packet scheduling unit that there is a new command data packet to be processed.

5. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, It also includes the step of prefetching the data to be processed into a local buffer through the DMA engine embedded in the packet scheduling unit during the execution of the task by the data processing unit.

6. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, In a virtualized environment, the translation of the user virtual address to the physical address includes the following steps: In the outbound direction, the translation from virtual address to physical address is completed by the host processor's memory management unit; In the inbound direction, address translation is performed when the accelerator accesses memory through the input / output memory management unit.

7. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, The packet scheduling unit consists of a hardware scheduler and runtime software, with the runtime software running on a designated RISC-V processor core or a dedicated RISC-V microcore integrated within a data stream accelerator.

8. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, It also includes the step of setting a barrier data packet in the circular buffer, the barrier data packet being used to indicate that all command data packets preceding the barrier must be executed before subsequent data packets can continue to be executed.

9. The data processing task scheduling method for a data stream accelerator as described in claim 1, characterized in that, The step of resetting the signal associated with the marked invalid command data packet includes setting a completion flag in a shared memory location to wake up a host application waiting for the task to complete.

10. A data stream accelerator system, characterized in that, include: The host processor, a RISC-V architecture processor core, is used to execute applications and generate data processing tasks; A data stream accelerator, connected to the host processor, includes multiple hardware data processing components for performing at least one of the following operations: data comparison, verification, migration, format conversion, and protocol offloading. The packet scheduling unit, including a hardware scheduling engine and runtime software, is connected to the host processor and the data stream accelerator and is used to schedule data processing tasks. Shared memory, including a circular buffer, for storing command data packets written by the host processor; A doorbell register or shared memory location is used to receive doorbell signals sent by the host processor; The address translation unit, including the memory management unit and the input / output memory management unit, is used to perform virtual-to-physical address translation in a virtualized environment; The packet scheduling unit is configured as follows: Receive the queue context initialized by the host processor through the runtime software; Assign packet identifiers and user virtual addresses based on the queue context; In response to the doorbell signal, the user's virtual address is converted into a physical address based on the base address and the read index; Parse the command data packet at the physical address to determine the target data processing unit and operation type; The task is scheduled to be executed by the corresponding data processing component. After the task is completed, update the read index and reset the completion signal.

Images