Dynamic task scheduling method and system for hybrid neural network

Abstract

The invention discloses a hybrid neural network-oriented dynamic task scheduling method and system, which is applied to the field of task scheduling of neural network accelerators, and comprises the following steps: performing subtask decomposition on a calculation task of a current neural network layer, constructing a task scheduling table containing preposed task state information based on task dependency information, and performing task scheduling on the task scheduling table; the state is dynamically updated by monitoring a completion event notification in the execution process, so that on-demand triggering and iterative scheduling of the sub-tasks are realized; when any subtask of the current layer is executed, if it is detected that the data loading instruction of the next neural network layer is stored in the corresponding instruction queue, a data preloading instruction is generated immediately, and the data loading operation of the next layer and calculation of the current layer are executed in parallel. According to the dynamic task scheduling method for the hybrid neural network provided by the invention, dynamic scheduling and cross-layer pipeline overlapping during real operation are realized, and the hardware resource utilization rate and the system throughput efficiency are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology for neural network accelerators, and in particular to a dynamic task scheduling method and system for hybrid neural networks. Background Technology

[0002] With the rapid development of artificial intelligence technology, hybrid neural network models such as convolutional neural networks (CNNs) and transformers have been widely used in fields such as computer vision and natural language processing. These hybrid models typically have complex computational structures and diverse data accuracy requirements, which place comprehensive demands on hardware accelerators, requiring high throughput, low latency, and high energy efficiency.

[0003] Existing neural network accelerators employ a static task scheduling strategy, where the task execution order and dependencies are determined before execution. Under this strategy, data loading operations for each neural network layer must wait for the completion of all computational tasks for that layer before they can begin. This makes it difficult for subsequent functional units to operate continuously in the pipeline, resulting in periodic idle hardware resources and limiting further improvements in system throughput. Summary of the Invention

[0004] This invention provides a dynamic task scheduling method and system for hybrid neural networks to solve the technical problem that data loading and computation operations cannot overlap and hardware resource utilization is limited due to the lack of runtime flexibility in task scheduling during the execution of neural network layers. This achieves the effects of improving task-level parallelism, reducing pipeline idle cycles, and improving the overall throughput efficiency of the system.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a dynamic task scheduling method for hybrid neural networks, comprising: The computational task of the current neural network layer is decomposed into sub-tasks; A task scheduling table is constructed based on the task dependency information corresponding to the computing task. The task dependency information defines the data dependency relationship between the subtasks. The task scheduling table records the preceding task status information of the subtask. The scheduling process based on the task scheduling table includes: determining the target subtask whose status information of the preceding task is in a completed state, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; Listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to a completed status based on the completion event notifications; The scheduling process is executed iteratively based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship. When executing any of the subtasks of the current neural network layer, determine whether the data loading subtask of the next neural network layer meets the pre-triggering condition based on the task dependency information; When the data loading subtask meets the triggering condition, a data preloading instruction is generated before the subtask of the current neural network layer is completed, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

[0006] As one preferred embodiment, the decomposition of the computational task of the current neural network layer into sub-tasks includes: Obtain the task scheduling instruction corresponding to the computational task of the current neural network layer; Based on the opcode of each task scheduling instruction, determine the subtask type to which the task scheduling instruction belongs; The task scheduling instructions are distributed to the corresponding instruction queues according to the corresponding subtask types to obtain each subtask.

[0007] As one preferred embodiment, the step of constructing a task scheduling table based on task dependency information corresponding to the computation task includes: Based on the task dependency information, a corresponding preset target value is set for each of the subtasks. The preset target value represents the number of times the completion event notification needs to be received before the subtask is executed. Initialize a current count value for each of the subtasks, the current count value being used to accumulate the number of times the completion event notification is received; The current count value and the corresponding preset target value are used as the status information of the preceding task to construct the task scheduling table.

[0008] As one preferred embodiment, the step of listening to and receiving a completion event notification corresponding to the target subtask, and updating the target subtask in the task scheduling table to a completed status based on the completion event notification, includes: Based on the completion event notification, increment the current count value of the target subtask in the task scheduling table by one; When the current count value is equal to the preset target value corresponding to the target subtask, the target subtask is updated to the completed status in the task scheduling table.

[0009] As one preferred embodiment, determining whether the data loading subtask of the next neural network layer satisfies the pre-triggering condition based on the task dependency information includes: During the execution of any of the subtasks of the current neural network layer, it is detected whether the data loading instruction for the next neural network layer has been stored in the corresponding instruction queue. If the data loading instruction is stored in the corresponding instruction queue, then it is determined that the data loading subtask meets the pre-triggering condition.

[0010] Another embodiment of the present invention provides a dynamic task scheduling system for hybrid neural networks, comprising: The task decomposition module is used to decompose the computational task of the current neural network layer into subtasks; The scheduling table construction module is used to construct a task scheduling table based on the task dependency information corresponding to the computing task. The task dependency information defines the data dependency relationship between the subtasks, and the task scheduling table records the previous task status information of the subtask. The scheduling and execution module is used to execute a scheduling process based on the task scheduling table, including: determining the target subtask whose status information of the preceding task is in a completed state, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; The status update module is used to listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to the completed status according to the completion event notifications. The iterative control module is used to iteratively execute the scheduling process based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship. The pre-triggering judgment module is used to determine, based on the task dependency information, whether the data loading subtask of the next neural network layer meets the pre-triggering condition when executing any of the subtasks of the current neural network layer. The preloading instruction generation module is used to generate a data preloading instruction before the completion of the subtask of the current neural network layer when the data loading subtask meets the triggering condition, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

[0011] As one preferred embodiment, the task decomposition module includes: The instruction acquisition unit is used to acquire the task scheduling instruction corresponding to the computation task of the current neural network layer. The type determination unit is used to determine the subtask type to which the task scheduling instruction belongs based on the opcode of each task scheduling instruction; The classification unit is used to distribute the task scheduling instructions to the corresponding instruction queues according to the corresponding subtask types, thereby obtaining each subtask.

[0012] As one preferred embodiment, the scheduling table construction module includes: The target value preset unit is used to set a corresponding preset target value for each of the subtasks according to the task dependency information. The preset target value represents the number of times the completion event notification needs to be received before the subtask is executed. An initialization unit is used to initialize a current count value for each of the subtasks, wherein the current count value is used to accumulate the number of times the completion event notification is received; The construction unit is used to construct the task scheduling table by taking the current count value and the corresponding preset target value as the previous task status information.

[0013] As one preferred embodiment, the state update module includes: An operation unit is configured to increment the current count value of the target subtask in the task scheduling table by one according to the completion event notification. The data update unit is used to update the target subtask to a completed state in the task scheduling table when the current count value is equal to the preset target value corresponding to the target subtask.

[0014] As one preferred embodiment, the pre-trigger judgment module includes: The detection unit is used to detect whether the data loading instruction for the next neural network layer is stored in the corresponding instruction queue during the execution of any of the subtasks of the current neural network layer. The judgment unit is used to determine that the data loading subtask satisfies the pre-triggering condition if the data loading instruction is stored in the corresponding instruction queue.

[0015] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following: (1) This invention decomposes the computational task of the current neural network layer into subtasks, constructs a task scheduling table containing the state information of the preceding tasks based on task dependency information, and dynamically updates the state by listening for completion event notifications during execution, thereby realizing on-demand triggering and iterative scheduling of subtasks; furthermore, during the execution of any subtask in the current layer, if it is detected that the data loading instruction of the next neural network layer has been stored in the corresponding instruction queue, a data preloading instruction is immediately generated, so that the data loading operation of the next layer is executed in parallel with the computation of the current layer. Compared with the static pipeline scheduling commonly used in the prior art, this invention abandons the forced waiting for the completion of the entire layer, and instead uses event-driven fine-grained state awareness as the basis to realize dynamic scheduling and cross-layer pipeline overlap during actual runtime, which significantly improves hardware resource utilization and system throughput efficiency.

[0016] (2) The dynamic task scheduling method for hybrid neural networks of the present invention significantly improves the resource utilization efficiency and end-to-end execution performance of neural network accelerators. On the one hand, by integrating task decomposition, dependency tracking and event feedback closed loop into the control module, different types of subtasks can be truly executed in parallel on dedicated hardware units, effectively eliminating pipeline bubbles caused by coarse-grained synchronization between modules; on the other hand, a cross-layer pre-triggering mechanism is introduced to start the data loading operation of the next layer in advance before the current neural network layer is completed, making full use of the orthogonality of data loading and computation operations on hardware resources and masking external storage access latency. Especially for hybrid precision convolutional neural networks and transformer models deployed on edge devices, this solution can efficiently support multi-granularity quantization strategies and complex computation modes under a unified architecture without modifying the model structure or inserting explicit synchronization points, taking into account high throughput, low power consumption and strong versatility. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a dynamic task scheduling method for hybrid neural networks in one embodiment of the present invention. Figure 2 This is a schematic diagram of a neural network accelerator system supporting mixed-precision computing in one embodiment of the present invention; Figure 3 This is a schematic diagram of a dynamic task scheduling system for hybrid neural networks according to one embodiment of the present invention.

[0018] Figure label: Among them, 11. Task decomposition module, 12. Schedule table construction module, 13. Schedule execution module, 14. Status update module, 15. Iteration control module, 16. Pre-trigger judgment module, and 17. Preload instruction generation module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0021] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0022] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0023] This invention relates to a neural network accelerator system supporting mixed-precision computing. The accelerator includes: a control module (Ctrl), an instruction fetch module (Fetch), a data loading module (Load), a matrix computation module (GEMM), a nonlinear operation module (ALU), and a data storage module (Store). Each functional module is connected via a shared bus or a dedicated interconnect network and is equipped with an independent instruction queue.

[0024] The control module is responsible for parsing the instruction stream of the neural network layers and distributing the instructions to the corresponding module's instruction queue according to the opcode. Simultaneously, the control module maintains a task scheduling table to track the dependency status of each subtask. The instruction acquisition module is responsible for reading the instruction stream from external memory and distributing the instructions to the corresponding module's instruction queue according to the opcode. After completing their tasks, each functional module sends a completion event notification to the upstream dependency queue; the control module listens for these notifications, updates the scheduling table accordingly, and then sends execution event tokens to downstream modules to trigger the next stage of operation.

[0025] Based on this hardware, one embodiment of the present invention provides a dynamic task scheduling method for hybrid neural networks. For details, please refer to [link to relevant documentation]. Figure 1 , Figure 1 The diagram shown is a flowchart of a dynamic task scheduling method for hybrid neural networks according to one embodiment of the present invention, which includes steps S1 to S7: S1: Decompose the computational task of the current neural network layer to obtain each subtask; Decomposing the computational task of the current neural network layer into multiple subtasks is to break down complex hierarchical computations into several basic operational units that can be executed independently on dedicated hardware functional modules. Since hybrid neural networks typically include multiple computational modes such as convolution, self-attention, and nonlinear activation, processing them uniformly is difficult to optimize hardware efficiency. Task decomposition maps different types of operations to corresponding dedicated execution units, laying the foundation for subsequent parallel scheduling, reducing inter-module coupling, and improving overall pipeline throughput.

[0026] Preferably, in one embodiment of the present invention, the computational task of the current neural network layer is decomposed into various sub-tasks, including: Obtain the task scheduling instruction corresponding to the computation task of the current neural network layer; Based on the opcode of each task scheduling instruction, determine the subtask type to which the task scheduling instruction belongs; The task scheduling instructions are distributed to the corresponding instruction queues according to the corresponding subtask types, thus obtaining each subtask.

[0027] Specifically, the instruction fetch module retrieves the instruction sequence corresponding to the current neural network layer from external memory or the instruction cache to obtain the task scheduling instruction corresponding to the computation task. Each instruction is a fixed-length binary code, with the high 4 bits representing the opcode field. The opcode field value corresponds one-to-one with the subtask type; for example, opcode 0001 represents the data loading subtask, 0010 represents the matrix multiplication calculation subtask, 0011 represents the nonlinear operation subtask, and 0100 represents the data storage subtask. Common subtask types include data loading subtasks, matrix multiplication calculation subtasks, nonlinear operation subtasks, and data storage subtasks. Specifically, the data loading subtask reads weight data or inputs activation values ​​from external dynamic random access memory (DRAM) and writes them to on-chip static random access memory (SRAM); the matrix multiplication calculation subtask performs mixed-precision matrix multiplication accumulation operations on the reconfigurable processing unit array; the nonlinear operation subtask performs activation function, shift, or element-wise addition operations on the accumulation result; and the data storage subtask writes the final output result from the on-chip SRAM back to external DRAM.

[0028] After determining the subtask type of each task scheduling instruction, the instruction retrieval module writes the instruction into a dedicated instruction queue bound to the subtask type. Specifically, data loading instructions are written to the loading instruction queue, which is an 8-depth FIFO buffer, with its output connected to the instruction interface of the data loading module; matrix multiplication calculation instructions are written to the calculation instruction queue, which is a 4-depth FIFO, with its output connected to the instruction decoder of the GEMM processing unit. Each functional module can independently retrieve instructions from its corresponding instruction queue and execute them without waiting for other types of instructions to complete.

[0029] By separating instructions according to operation type, different types of subtasks can be assigned to dedicated hardware units for parallel execution, which can fully leverage the heterogeneous computing capabilities of the accelerator. In addition, the instruction queue, as a task buffer, decouples the timing dependency between instruction fetching and task execution, providing a foundation for subsequent event-driven dynamic scheduling.

[0030] S2: Construct a task scheduling table based on the task dependency information corresponding to the computation task. The task dependency information defines the data dependency relationship between subtasks, and the task scheduling table records the status information of the preceding tasks of the subtasks. A task scheduling table is constructed based on the task dependency information corresponding to the computational tasks of the current neural network layer. Its purpose is to dynamically track the completion status of the prerequisite dependencies of each subtask in a quantitative manner, thereby providing a clear triggering basis for runtime scheduling and avoiding pipeline blocking problems caused by global barrier synchronization mechanisms. Task dependency information refers to control metadata describing the data production and consumption relationships between subtasks within the current neural network layer. For example, in a typical convolutional or Transformer layer, the execution of the matrix multiplication subtask depends on the completion of one or more data loading subtasks, while the nonlinear operation subtask depends on the output of the matrix multiplication subtask. This information is generated by the compiler or configuration tool during the neural network model deployment phase and loaded into the accelerator's control module along with the task scheduling instructions. The task scheduling table is a data structure within the control module used to track the dependency status of each subtask in real time. Its entity form is a set of dedicated status registers, maintaining two key fields for each subtask in the current neural network layer: a preset target value and a current count value.

[0031] Preferably, in one embodiment of the present invention, constructing a task scheduling table based on task dependency information corresponding to the computing task includes: Based on the task dependency information, set corresponding preset target values ​​for each subtask. The preset target value indicates the number of times a completion event notification needs to be received before the subtask is executed. Initialize the current count value for each subtask. The current count value is used to accumulate the number of times a completion event notification has been received. The current count value and the corresponding preset target value are used as the status information of the preceding tasks to construct a task scheduling table.

[0032] The preset target value is set during the task initialization phase based on the task dependency information mentioned above. It represents the number of completion event notifications that must be received before the subtask is allowed to be executed. The current count value is dynamically updated during runtime. The initial value is zero, and the value is incremented by one whenever the control module receives a completion event notification related to the subtask from the uplink dependency queue.

[0033] Specifically, after receiving all task scheduling instructions for the current neural network layer, the control module first parses the dependency requirements of each subtask. For each subtask, the control module assigns a preset target value. For example, if a matrix multiplication subtask requires two independent data loading operations to complete, its preset target value is set to 2; if a nonlinear operation subtask depends on only one matrix calculation operation, its preset target value is set to 1. Then, the control module allocates a corresponding current count value for each subtask in its internal register file and initializes it to zero. This current count value is used to accumulate the number of completion event notifications received during runtime that are related to that subtask. Whenever the control module receives a completion event notification from the upstream dependency queue, it parses the target subtask identifier carried in the notification and increments the current count value of the corresponding subtask by one.

[0034] In the scheduling process, the control module determines whether the status information of the prerequisite tasks for a subtask has met the execution conditions by comparing the current count value with the preset target value. When the two are equal, it indicates that all prerequisites are ready, and the subtask is marked as schedulable.

[0035] Replacing traditional explicit dependency graph traversal or barrier signals with counters significantly reduces the hardware complexity and decision latency of the scheduling logic. Furthermore, since completion event notifications are proactively issued by functional modules at the end of a task, scheduling decisions are entirely based on the actual execution progress, achieving true on-the-fly dynamic scheduling. This method, which supports many-to-one dependency scenarios, eliminates the need for additional dependency parsing circuitry, enhancing the accelerator's adaptability to hybrid neural network structures.

[0036] S3: Execute the scheduling process based on the task scheduling table, including: determining the target subtask whose status information of the preceding task is completed, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; S4: Listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to the completed status based on the completion event notifications; In this embodiment, steps S3 and S4 together constitute an event-driven dynamic scheduling execution closed loop, which avoids the resource idleness problem caused by static pipelines or global synchronization mechanisms. By sensing the task completion status in real time at runtime, it realizes on-demand triggering and status feedback of subtasks, thereby improving the utilization of hardware resources and system throughput efficiency. Specifically, the prerequisite task status information refers to the data status used to characterize whether all prerequisite dependencies of a subtask have been satisfied; the completion event notification is a control signal actively issued by the functional module after the subtask is completed, used to provide feedback on the task completion status to the control module.

[0037] Preferably, in one embodiment of the present invention, monitoring and receiving a completion event notification corresponding to the target subtask, and updating the target subtask in the task scheduling table to a completed state according to the completion event notification, includes: Based on the completion event notification, increment the current count value of the corresponding target subtask in the task scheduling table by one; When the current count value equals the preset target value corresponding to the target subtask, the target subtask will be updated to the completed status in the task scheduling table.

[0038] Specifically, the scheduling process periodically traverses the entries corresponding to each subtask in the task scheduling table using a polling method, checking whether the current count value of each subtask is equal to its preset target value. If they are equal, it indicates that all prerequisites of the subtask have been satisfied, its prerequisite task status information is complete, and the subtask is identified as the target subtask. The control module then generates corresponding execution instructions based on the type of the target subtask and its associated instruction content. The execution instructions include operation parameters, data addresses, and functional module identifiers, and are sent to the corresponding functional module through the downlink dependency queue. For example, if the target subtask is a matrix multiplication calculation subtask, the execution instructions are written to the instruction interface of the GEMM processing unit; if it is a nonlinear operation subtask, it is sent to the ALU module. Upon receiving the execution instructions, the functional module immediately starts the corresponding operation.

[0039] During this period, the control module continuously monitors the uplink dependency queue to receive completion event notifications from various functional modules. In this embodiment, each functional module automatically generates a completion event notification after its subtask is completed and writes it to the uplink dependency queue. The completion event notification is a data packet with a fixed format, containing a target subtask identifier and an event type field. When the control module reads the notification from the uplink dependency queue, it parses the subtask identifier, locates the corresponding entry in the task scheduling table, and increments the current count value of that entry. When the updated current count value equals the preset target value of the subtask, the control module marks the subtask's status in the task scheduling table as completed, indicating that its output data is ready and can be used as the input source for subsequent subtasks.

[0040] In one embodiment of the present invention, GEMM may employ a Dynamic Precision Processing Element (DP-PE) as its core computing array. The DP-PE supports various quantization precisions, including 2-bit, 4-bit, 8-bit, and 16-bit, and achieves dynamic reuse of hardware resources through a hierarchical recursive architecture.

[0041] Each GEMM task scheduling instruction includes a 2-bit precision control field. This field is written into the computation instruction queue by the instruction fetching module during the instruction distribution phase and parsed by the internal instruction decoding logic of the GEMM module during execution. It is used to configure the data path mode of each level of basic computation modules in DP-PE. Although DP-PE exhibits different parallelism and data flow structures in different precision modes (e.g., reconstructed into 8 parallel multipliers in 2-bit mode, and as a single high-precision multiply-accumulate unit in 16-bit mode), its external interaction behavior remains consistent: the initiation of GEMM subtasks is always triggered by the control module sending an execution event token through the downlink dependency queue; after the task is completed, the GEMM module also returns a uniformly formatted completion event notification to the control module through the uplink dependency queue. This notification does not carry precision information, but only indicates that "a GEMM operation has been completed". In order to match the high parallel output of DP-PE in low-precision mode, the GEMM module also integrates a reconfigurable accumulator, which can dynamically adjust the accumulation channel width according to the same precision control signal to ensure that the data reduction phase does not become a performance bottleneck.

[0042] Employing DP-PE makes the dynamic task scheduling method in this embodiment completely transparent to the underlying computational precision. Regardless of the mixed precision strategy used by the current neural network layer, the control module does not need to be aware of or adapt to the internal working mode of DP-PE; it can make scheduling decisions solely based on task dependencies and event notifications. This feature enables this solution to seamlessly support the multi-granularity quantization requirements common in hybrid CNN and Transformer models (such as using 8-bit for attention weights and 4-bit for FFN layers). While improving hardware energy efficiency, it avoids the scheduling complexity introduced by precision switching, ensuring the simplicity, versatility, and efficiency of the scheduling mechanism.

[0043] In the dynamic scheduling execution closed loop of this embodiment, the scheduling strategy is based entirely on the actual execution progress, rather than a preset timing sequence. This allows subtasks with different latency characteristics (such as Load and GEMM) to be naturally aligned, avoiding the blocking of fast paths due to waiting for slow operations. Secondly, the use of a counter mechanism instead of explicit dependency graph management significantly simplifies the hardware implementation complexity of the control logic. This closed-loop scheduling process supports subtask dependencies with arbitrary topologies, making it particularly suitable for complex multi-branch data flow scenarios in hybrid CNN and Transformer models, significantly enhancing the accelerator's versatility and adaptability.

[0044] S5: Iteratively execute the scheduling process based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship; In this embodiment, the scheduling process is iteratively executed based on the updated task scheduling table to ensure that all subtasks within the current neural network layer are completed in an orderly manner according to the data dependencies determined during the compilation phase, while avoiding pipeline idleness or resource contention issues caused by static scheduling strategies. This iterative mechanism is the core element for achieving dynamic, closed-loop scheduling, enabling the scheduler to continuously respond to runtime state changes and trigger executable subtasks as needed.

[0045] Specifically, after the control module completes a scheduling operation (including target subtask identification, execution instruction generation and issuance) and receives at least one completion event notification, the current count value in the task scheduling table is updated. At this time, the control module immediately starts the next scheduling loop, re-traversing all subtask entries in the task scheduling table. For each subtask, the control module compares its current count value with the preset target value again: if they are equal and the subtask has not yet been executed, it is identified as a new target subtask, and the corresponding execution instruction is generated; if they are not equal or have already been executed, it is skipped. This traversal and judgment process is driven by the state machine inside the control module with a fixed clock cycle, forming continuous scheduling rounds. According to the hardware architecture on which this embodiment is based, this iterative process does not require external intervention or global synchronization signals. Specifically, the control module continuously monitors the status of the task scheduling table through polling and, in conjunction with the completion event notifications in the uplink dependency queue, autonomously decides whether to enter a new round of scheduling. When it is detected that all subtasks of the current neural network layer have met the condition that "the current count value equals the preset target value" and the execution instruction has been successfully issued, the control module determines that the computation task of that layer has been completed and sends a completion signal to the host processor or the next layer of scheduling logic.

[0046] S6: When executing any subtask of the current neural network layer, determine whether the data loading subtask of the next neural network layer meets the pre-triggering condition based on the task dependency information; The pre-triggering condition refers to the criterion used to determine whether the data loading subtask of the next neural network layer can be started in advance, even before the current neural network layer has been completed. Specifically, in this embodiment, the pre-triggering condition is defined as the data loading instruction for the next neural network layer having been stored in the corresponding loading instruction queue. Since the data address, data length, and target storage location required by the data loading subtask of the next neural network layer are determined during the compilation stage and encoded in the operand field of the loading instruction, the execution of this subtask does not depend on any calculation result of the current layer. As long as the loading instruction successfully enters the loading instruction queue, it indicates that all its execution prerequisites have been met, and it is feasible to execute immediately.

[0047] Preferably, in one embodiment of the present invention, determining whether the data loading subtask of the next neural network layer satisfies the pre-triggering condition based on task dependency information includes: During the execution of any subtask of the current neural network layer, check whether the data loading instruction for the next neural network layer has been stored in the corresponding instruction queue. If the data loading instruction is stored in the corresponding instruction queue, then the data loading subtask is deemed to meet the pre-triggering condition.

[0048] In the reconfigurable neural network accelerator architecture used in this embodiment, the instruction streams for each neural network layer are sequentially issued by the host processor or on-chip instruction scheduler. When the control module begins processing the computation task of the current neural network layer, it first parses the instructions for the current layer and distributes various subtask instructions to corresponding dedicated instruction queues, including load instruction queues, computation instruction queues, nonlinear operation instruction queues, and storage instruction queues. Simultaneously, the instructions for the next neural network layer may have arrived partially or completely and been prefetched into the instruction cache. During the execution of any subtask of the current layer (e.g., the GEMM computation subtask), the control module continuously monitors the status of the load instruction queue. If it detects that the data loading instruction for the next neural network layer has been successfully written into the load instruction queue, it determines that the data loading subtask meets the pre-triggering condition.

[0049] Since the data loading subtask for the next neural network layer does not depend on any computational output of the current layer, its required data address and length information are determined during the model compilation phase and provided directly through instructions. Therefore, as long as the loading instruction has entered the loading instruction queue, it indicates that all the control information required for its execution is ready. There is no need to wait for the current layer to complete; that is, the control module does not need to check whether the loading subtask has any prerequisites, nor does it need to wait for the completion event notification of the current layer. It only needs to confirm that the instruction has been enqueued to trigger the loading.

[0050] In this embodiment, the orthogonality of data loading and computation operations in terms of hardware resources allows data loading for the next layer to begin before the computation of the current layer has finished. Since data loading is performed by a separate DMA module, while computation is handled by a GEMM or ALU unit, both using different hardware resources, they can work in parallel, effectively reducing idle computation units caused by waiting for data transfer. Furthermore, the pre-triggering judgment only needs to check if the loading instruction has entered the queue; this operation only requires reading a single status bit in the control module, making it simple to implement and adding almost no scheduling latency. Especially for model structures like CNNs or Transformers where inter-layer data is independent, this method can naturally achieve cross-layer pipeline overlap without any model modification or additional synchronization instructions, improving overall execution efficiency.

[0051] S7: When the data loading subtask meets the triggering condition, a data preloading instruction is generated before the subtask of the current neural network layer is completed, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

[0052] Before the subtask of the current neural network layer is completed, a data preloading instruction is generated to trigger the data loading subtask of the next neural network layer to be executed in parallel with the subtask of the current layer. The purpose is to overcome the problem of alternating idleness of computing units and memory subsystem caused by traditional inter-layer serial scheduling. By starting the data transfer operation of the next layer in advance, the overlapping execution of computation and memory access is achieved, thereby improving the overall utilization of hardware resources.

[0053] Specifically, in the reconfigurable neural network accelerator architecture adopted in this embodiment, the control module continuously monitors the status of the loading instruction queue during the execution of any subtask of the current neural network layer (e.g., matrix multiplication calculation subtask). When it is detected that the data loading instruction for the next neural network layer has been successfully written into the loading instruction queue, the control module immediately determines that the data loading subtask meets the pre-triggering condition. At this time, even though the current neural network layer has not yet completed all subtasks, the control module will still generate a data preloading instruction and send the instruction to the data loading module through the downlink dependency queue. After receiving the instruction, the data loading module immediately initiates a direct memory access (DMA) operation to read the weight data or activation values ​​required by the next neural network layer from the external dynamic random access memory and cache them in the on-chip static random access memory.

[0054] Because the parameters such as data address, data length, and target storage location required for the data loading subtask of the next neural network layer are determined during the compilation phase and fully encoded in the operand field of the loading instruction, its execution does not depend on any calculation results of the current layer. Therefore, as long as the loading instruction is ready (i.e., stored in the loading instruction queue), execution can be safely triggered without causing data conflicts or logical errors. In this embodiment, data loading is performed by a dedicated DMA engine, while the calculation of the current layer is performed by the GEMM or ALU processing unit. The two use independent hardware resources, thus enabling true parallel operation and effectively masking memory access latency. On the other hand, since the generation of the preloading instruction only depends on the simple state judgment of whether the loading instruction is enqueued, its implementation only requires adding a queue non-empty detection logic to the control module, resulting in minimal hardware overhead and no interference with the normal scheduling process of the current layer. Especially for model structures such as convolutional neural networks or Transformers where inter-layer data is independent, this embodiment does not require inserting additional synchronization barriers or modifying the model computation graph, and can naturally achieve continuous filling of the cross-layer pipeline, significantly reducing pipeline bubbles and improving the overall execution efficiency of the accelerator.

[0055] In this embodiment, the above mechanism is achieved based on a highly modular and loosely coupled accelerator architecture. This architecture not only supports fine-grained parallelism of subtasks within layers, but also breaks down execution barriers between layers through a cross-layer pre-triggering mechanism. This allows for efficient handling of the complex computational load of hybrid convolutional neural network and transformer models on a unified hardware platform, balancing high throughput, low latency, and high resource utilization. For details, see [link to documentation]. Figure 2 , Figure 2 The diagram shows a schematic of a neural network accelerator system supporting mixed-precision computing, on which an embodiment of the present invention is based.

[0056] Another embodiment of the present invention provides a dynamic task scheduling system for hybrid neural networks. For details, please refer to [link to relevant documentation]. Figure 3 , Figure 3 The diagram shown illustrates a dynamic task scheduling system for hybrid neural networks according to one embodiment of the present invention, which includes: Task decomposition module 11 is used to decompose the computational task of the current neural network layer into sub-tasks; The scheduling table construction module 12 is used to construct a task scheduling table based on the task dependency information corresponding to the computing task. The task dependency information defines the data dependency relationship between subtasks, and the task scheduling table records the status information of the preceding tasks of the subtasks. The scheduling execution module 13 is used to execute the scheduling process based on the task scheduling table, including: determining the target subtask whose status information of the preceding task is completed, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; The status update module 14 is used to listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to the completed status according to the completion event notifications. Iteration control module 15 is used to iteratively execute the scheduling process based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship; The pre-trigger judgment module 16 is used to determine, based on task dependency information, whether the data loading subtask of the next neural network layer meets the pre-trigger condition when executing any subtask of the current neural network layer. The preloading instruction generation module 17 is used to generate a data preloading instruction before the completion of the subtask of the current neural network layer when the data loading subtask meets the triggering condition, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

[0057] Preferably, in one embodiment of the present invention, the task decomposition module includes: The instruction acquisition unit is used to acquire the task scheduling instruction corresponding to the computation task of the current neural network layer. The type determination unit is used to determine the subtask type to which a task scheduling instruction belongs based on the opcode of each task scheduling instruction. The classification unit is used to distribute task scheduling instructions to the corresponding instruction queues according to the corresponding subtask types, thereby obtaining each subtask.

[0058] Preferably, in one embodiment of the present invention, the scheduling table construction module includes: The target value preset unit is used to set corresponding preset target values ​​for each subtask based on task dependency information. The preset target value represents the number of times a completion event notification needs to be received before the subtask is executed. The initialization unit is used to initialize the current count value for each subtask. The current count value is used to accumulate the number of times the completion event notification has been received. The construction unit is used to construct a task scheduling table by taking the current count value and the corresponding preset target value as the status information of the preceding task.

[0059] Preferably, in one embodiment of the present invention, the state update module includes: The operation unit is used to increment the current count value of the corresponding target subtask in the task scheduling table according to the completion event notification. The data update unit is used to update the target subtask to a completed state in the task scheduling table when the current count value is equal to the preset target value corresponding to the target subtask.

[0060] Preferably, in one embodiment of the present invention, the pre-trigger judgment module includes: The detection unit is used to detect whether the data loading instruction for the next neural network layer has been stored in the corresponding instruction queue during the execution of any subtask of the current neural network layer. The judgment unit is used to determine whether the data loading subtask meets the pre-triggering condition if the data loading instruction is stored in the corresponding instruction queue.

[0061] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following: (1) This invention significantly simplifies the hardware implementation complexity of scheduling control logic by transforming task dependencies into a quantifiable count value and a target value comparison mechanism. Compared with existing methods that require maintaining explicit dependency graphs or ready queues, this invention only requires a set of lightweight counters and simple numerical comparators to complete the state judgment. This not only reduces the area overhead and power consumption of the control module, but also improves the response speed of scheduling decisions, enabling the system to complete multi-event processing and multi-task dispatch within a single clock cycle, significantly enhancing the accelerator's real-time scheduling capability at high frequencies.

[0062] (2) The instruction distribution and task execution in this invention are completely decoupled. Each functional module independently obtains tasks through a dedicated instruction queue, avoiding the bottleneck caused by traditional centralized instruction decoding. Since the instruction acquisition module completes the task type division according to the opcode at the front end and pre-places the instructions into the corresponding queue, the central intelligent scheduling module does not need to participate in instruction parsing and only focuses on dependency state management and event scheduling, thereby achieving efficient separation of the control plane and the data plane and improving the throughput and scalability of the overall pipeline.

[0063] (3) The cross-layer data preloading mechanism in this invention effectively alleviates the performance bottleneck caused by the limited off-chip storage bandwidth in edge devices. By initiating the loading operation of the next layer's weights or activation values ​​in advance during the computation of the current neural network layer, the high-latency dynamic random access memory access and the low-latency on-chip computation overlap, significantly reducing the idle cycles of the computing unit due to waiting for data. This mechanism is particularly suitable for scenarios where self-attention and feedforward networks are executed alternately in the converter model, significantly improving end-to-end inference efficiency.

[0064] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A dynamic task scheduling method for hybrid neural networks, characterized in that, include: The computational task of the current neural network layer is decomposed into sub-tasks; A task scheduling table is constructed based on the task dependency information corresponding to the computing task. The task dependency information defines the data dependency relationship between the subtasks. The task scheduling table records the preceding task status information of the subtask. The scheduling process based on the task scheduling table includes: determining the target subtask whose status information of the preceding task is in a completed state, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; Listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to a completed status based on the completion event notifications; The scheduling process is executed iteratively based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship. When executing any of the subtasks of the current neural network layer, determine whether the data loading subtask of the next neural network layer meets the pre-triggering condition based on the task dependency information; When the data loading subtask meets the triggering condition, a data preloading instruction is generated before the subtask of the current neural network layer is completed, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

2. The dynamic task scheduling method for hybrid neural networks as described in claim 1, characterized in that, The computational task of the current neural network layer is decomposed into sub-tasks, including: Obtain the task scheduling instruction corresponding to the computational task of the current neural network layer; Based on the opcode of each task scheduling instruction, determine the subtask type to which the task scheduling instruction belongs; The task scheduling instructions are distributed to the corresponding instruction queues according to the corresponding subtask types to obtain each subtask.

3. The dynamic task scheduling method for hybrid neural networks as described in claim 1, characterized in that, The step of constructing a task scheduling table based on the task dependency information corresponding to the computing task includes: Based on the task dependency information, a corresponding preset target value is set for each of the subtasks. The preset target value represents the number of times the completion event notification needs to be received before the subtask is executed. Initialize a current count value for each of the subtasks, the current count value being used to accumulate the number of times the completion event notification is received; The current count value and the corresponding preset target value are used as the status information of the preceding task to construct the task scheduling table.

4. The dynamic task scheduling method for hybrid neural networks as described in claim 3, characterized in that, The process of listening to and receiving completion event notifications corresponding to the target subtask, and updating the target subtask in the task scheduling table to a completed status based on the completion event notification, includes: Based on the completion event notification, increment the current count value of the target subtask in the task scheduling table by one; When the current count value is equal to the preset target value corresponding to the target subtask, the target subtask is updated to the completed status in the task scheduling table.

5. The dynamic task scheduling method for hybrid neural networks as described in claim 1, characterized in that, The step of determining whether the data loading subtask of the next neural network layer meets the pre-triggering condition based on the task dependency information includes: During the execution of any of the subtasks of the current neural network layer, it is detected whether the data loading instruction for the next neural network layer has been stored in the corresponding instruction queue. If the data loading instruction is stored in the corresponding instruction queue, then it is determined that the data loading subtask meets the pre-triggering condition.

6. A dynamic task scheduling system for hybrid neural networks, characterized in that, include: The task decomposition module is used to decompose the computational task of the current neural network layer into subtasks; The scheduling table construction module is used to construct a task scheduling table based on the task dependency information corresponding to the computing task. The task dependency information defines the data dependency relationship between the subtasks, and the task scheduling table records the previous task status information of the subtask. The scheduling and execution module is used to execute a scheduling process based on the task scheduling table, including: determining the target subtask whose status information of the preceding task is in a completed state, and generating an execution instruction for the target subtask to trigger the execution of the target subtask; The status update module is used to listen for and receive completion event notifications corresponding to the target subtask, and update the target subtask in the task scheduling table to the completed status according to the completion event notifications. The iterative control module is used to iteratively execute the scheduling process based on the updated task scheduling table until each subtask in the current neural network layer is completed according to the data dependency relationship. The pre-triggering judgment module is used to determine, based on the task dependency information, whether the data loading subtask of the next neural network layer meets the pre-triggering condition when executing any of the subtasks of the current neural network layer. The preloading instruction generation module is used to generate a data preloading instruction before the completion of the subtask of the current neural network layer when the data loading subtask meets the triggering condition, so as to trigger the data loading subtask and the subtask of the current neural network layer to be executed in parallel.

7. A dynamic task scheduling system for hybrid neural networks as described in claim 6, characterized in that, The task decomposition module includes: The instruction acquisition unit is used to acquire the task scheduling instruction corresponding to the computation task of the current neural network layer. The type determination unit is used to determine the subtask type to which the task scheduling instruction belongs based on the opcode of each task scheduling instruction; The classification unit is used to distribute the task scheduling instructions to the corresponding instruction queues according to the corresponding subtask types, thereby obtaining each subtask.

8. A dynamic task scheduling system for hybrid neural networks as described in claim 6, characterized in that, The scheduling table construction module includes: The target value preset unit is used to set a corresponding preset target value for each of the subtasks according to the task dependency information. The preset target value represents the number of times the completion event notification needs to be received before the subtask is executed. An initialization unit is used to initialize a current count value for each of the subtasks, wherein the current count value is used to accumulate the number of times the completion event notification is received; The construction unit is used to construct the task scheduling table by taking the current count value and the corresponding preset target value as the previous task status information.

9. A dynamic task scheduling system for hybrid neural networks as described in claim 8, characterized in that, The status update module includes: An operation unit is configured to increment the current count value of the target subtask in the task scheduling table by one according to the completion event notification. The data update unit is used to update the target subtask to a completed state in the task scheduling table when the current count value is equal to the preset target value corresponding to the target subtask.

10. A dynamic task scheduling system for hybrid neural networks as described in claim 6, characterized in that, The pre-trigger judgment module includes: The detection unit is used to detect whether the data loading instruction for the next neural network layer is stored in the corresponding instruction queue during the execution of any of the subtasks of the current neural network layer. The judgment unit is used to determine that the data loading subtask satisfies the pre-triggering condition if the data loading instruction is stored in the corresponding instruction queue.

Images