Task scheduling method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a task scheduling method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: receiving a tensor instruction distributed by a processor unit; querying the dependency check table, obtaining dependency check information of the tensor instruction, and performing dependency check according to the dependency check information; under the condition that the dependency check is passed, the tensor instruction is converted into a micro-operation control signal, and the micro-operation control signal is sent to the finite-state machine module, so that the finite-state machine module dispatches the processing unit module to execute tensor operation; and an execution completion signal fed back by the neural network acceleration unit after the tensor operation is completed is received, the dependency check table is updated, and an operation notification message is sent to the processor unit. By adopting the method, the processor unit can process other logic transactions in parallel or enter a low-power-consumption state while the neural network acceleration unit performs tensor operation, so that the overall throughput rate and the energy efficiency ratio of the heterogeneous system are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of integrated circuit design technology, and in particular to a task scheduling method, apparatus, computer equipment, readable storage medium, and program product. Background Technology

[0002] With the deep integration of artificial intelligence and Internet of Things technologies, the requirements for processors in edge computing scenarios have changed. In traditional heterogeneous architectures, general-purpose processors are responsible for logic control, while neural network acceleration units are responsible for intensive tensor operations.

[0003] However, this collaborative model has limitations in existing technical solutions: each computational task of the neural network processor requires the general-purpose processor to perform tedious register configuration, interrupt handling, and status polling. For modern neural networks with deep layers and fragmented operators, frequent interactions between the main processor and the neural network processor lead to excessive load on the main processor, and even situations where the main processor is busy scheduling while the neural network processor is idle and waiting. Summary of the Invention

[0004] Therefore, it is necessary to provide a task scheduling method, apparatus, computer device, readable storage medium, and program product that can realize in-memory parallel computing of processor units and neural network acceleration units to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a task scheduling method applied to a scheduling unit, wherein the scheduling unit is connected to a processor unit and a neural network acceleration unit, and the neural network acceleration unit includes a finite state machine module and multiple processing unit modules controlled by the finite state machine module. The method includes:

[0006] Receive tensor instructions distributed by the processor unit;

[0007] Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information;

[0008] If the dependency check passes, the tensor instruction is converted into a micro-operation control signal and sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0009] The system receives the execution completion signal from the neural network acceleration unit after completing the tensor operation, updates the dependency check table, and sends an operation notification message to the processor unit.

[0010] In one embodiment, the processor unit and the neural network acceleration unit share a memory space and use a unified memory address encoding;

[0011] The tensor instruction carries the memory address encoding, which points to the tensor data in the shared memory space;

[0012] The step of converting the tensor instruction into a micro-operation control signal includes:

[0013] The tensor instructions are converted into micro-operation control signals containing the memory address encoding, instructing the neural network acceleration unit to read tensor data from the shared memory space for computation according to the memory address encoding.

[0014] In one embodiment, the dependency check information of the tensor instruction includes a scalar tensor association marker and a memory address occupancy marker, wherein the scalar tensor association marker is used to characterize the association relationship between the scalar instruction and the tensor instruction; and the memory address occupancy marker is used to characterize the range of memory addresses that the processor unit or the neural network acceleration unit is operating on.

[0015] The dependency check based on the dependency check information includes suspending the tensor instruction in the hardware queue of the scheduling unit if the dependency check fails, until the dependency check passes.

[0016] In one embodiment, performing dependency checks based on the dependency check information includes:

[0017] Query the scalar tensor association flag to determine whether the operands of the tensor instruction depend on the scalar instruction that the processor unit has not yet completed;

[0018] The target read / write address range involved in the memory address encoding in the tensor instruction is parsed.

[0019] Query the memory address occupancy flag to determine whether the target read / write address range is locked by the incomplete write operation of the processor unit;

[0020] The dependency check fails if there are dependencies or if the device is locked.

[0021] In one embodiment, the processing unit module includes a boundary processing unit and an array processing unit;

[0022] Sending the micro-operation control signal to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations includes:

[0023] The micro-operation control signal is sent to the finite state machine module, instructing the finite state machine module to schedule the boundary processing unit to perform data format conversion or edge filling tasks;

[0024] And / or,

[0025] The micro-operation control signal is sent to the finite state machine module, instructing the finite state machine module to control the topology between the array processing units according to the micro-operation control signal, so as to reorganize multiple array processing units into a target computing array.

[0026] In one embodiment, the dependency check information includes a processing unit status flag, which is used to characterize the occupancy status and topology of the processing unit module;

[0027] The dependency check based on the dependency check information includes:

[0028] The target processing unit resource number and target topology required to parse the tensor instruction;

[0029] Query the status flag of the processing unit, and compare the resource number of the target processing unit with the occupied resource number and the corresponding current topology configuration recorded in the status flag of the processing unit;

[0030] The dependency check fails if there is overlap.

[0031] Secondly, this application also provides a task scheduling device integrated into a scheduling unit, wherein the scheduling unit is connected to a processor unit and a neural network acceleration unit, and the neural network acceleration unit includes a finite state machine module and multiple processing unit modules controlled by the finite state machine module. The device includes:

[0032] The instruction receiving module is used to receive tensor instructions distributed by the processor unit;

[0033] The dependency checking module is used to query the dependency checking table, obtain the dependency checking information of the tensor instruction, and perform dependency checks based on the dependency checking information.

[0034] The instruction dispatch module is used to convert the tensor instruction into a micro-operation control signal when the dependency check passes, and send the micro-operation control signal to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0035] The status feedback module is used to receive the execution completion signal fed back by the neural network acceleration unit after completing the tensor operation, update the dependency check table, and send an operation notification message to the processor unit.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Receive tensor instructions distributed by the processor unit;

[0038] Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information;

[0039] If the dependency check passes, the tensor instruction is converted into a micro-operation control signal and sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0040] The system receives the execution completion signal from the neural network acceleration unit after completing the tensor operation, updates the dependency check table, and sends an operation notification message to the processor unit.

[0041] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0042] Receive tensor instructions distributed by the processor unit;

[0043] Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information;

[0044] If the dependency check passes, the tensor instruction is converted into a micro-operation control signal and sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0045] The system receives the execution completion signal from the neural network acceleration unit after completing the tensor operation, updates the dependency check table, and sends an operation notification message to the processor unit.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Receive tensor instructions distributed by the processor unit;

[0048] Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information;

[0049] If the dependency check passes, the tensor instruction is converted into a micro-operation control signal and sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0050] The system receives the execution completion signal from the neural network acceleration unit after completing the tensor operation, updates the dependency check table, and sends an operation notification message to the processor unit.

[0051] The aforementioned task scheduling methods, apparatus, computer devices, readable storage media, and program products introduce an independent scheduling unit to take over the scheduling of tensor instructions. After distributing instructions, the processor unit can release resources to handle other tasks without software polling or blocking. The scheduling unit automatically handles dependency suspension and resource locking at the hardware level and proactively notifies the processor upon task completion. This mechanism enables the processor unit to process other logical tasks in parallel or enter a low-power state while the neural network acceleration unit performs tensor operations, significantly improving the overall throughput and energy efficiency of heterogeneous systems.

[0052] This application innovatively utilizes a dependency checklist as the core of hardware logic, enabling automatic dependency checks. Compared to traditional software bus scheduling, hardware checks offer extremely low latency. This mechanism can handle complex dependencies, such as scalar-tensor heterogeneous dependencies and address overlap dependencies. Instructions are only converted and dispatched if the dependency check passes; otherwise, they wait in the hardware queue, ensuring data consistency even under out-of-order or concurrent instruction scenarios.

[0053] This application provides a deep adaptation to an improved neural network acceleration unit architecture comprising a finite state machine module and a processing unit module. The scheduling unit does not directly control the underlying processing unit modules; instead, it sends micro-operation control signals to the finite state machine module, which then flexibly schedules the array of underlying processing unit modules. This hierarchical control, combined with resource dependency checks on the scheduling unit side, enables the system to support dynamic array reconfiguration and resource reuse, avoiding computational errors caused by resource conflicts, while maximizing the utilization of the neural network acceleration unit's computing array. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is an application architecture diagram of a task scheduling method in one embodiment;

[0056] Figure 2 This is a flowchart illustrating a task scheduling method in one embodiment;

[0057] Figure 3 This is a flowchart illustrating the dependency check step in one embodiment;

[0058] Figure 4 This is a flowchart illustrating the dependency check step in another embodiment;

[0059] Figure 5 This is a structural block diagram of a task scheduling device in one embodiment;

[0060] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0063] Traditional Intellectual Property (IP) core scheduling typically relies on the processor to perform software polling to query the status of the Neural Processing Unit (NPU), or the processor needs to block and wait for tensor operations to complete after distributing instructions. This prevents the processor from effectively handling other tasks (such as network interruptions or data preprocessing) during the NPU's operation, resulting in low overall system parallel efficiency.

[0064] As neural network processing unit architectures evolve, they typically contain complex control logic. Traditional schedulers often treat neural network processing units as black-box resources, unable to perceive their specific topology configuration or fine-grained resource usage. When faced with a mixed stream of scalar and tensor instructions, or multiple tasks competing for processing unit resources, traditional first-in-first-out scheduling strategies are prone to data hazards or resource conflicts, or excessive serialization to avoid conflicts, leading to idle computing resources.

[0065] Therefore, this invention proposes a hardware-based task scheduling scheme that can reduce the processor scheduling burden and perform efficient and accurate dependency checking on the complex internal architecture of the General-purpose Parallel Neural Processing Unit (GPNPU). Figure 1 As shown, the IP core system architecture used in this scheme mainly includes a processor unit, a scheduling unit, a neural network acceleration unit, and shared memory space. The scheduling unit is an independent hardware module located between the processor unit and the neural network acceleration unit. It is mainly responsible for sharing the processor's scheduling tasks. Internally, it maintains a dependency check table to record the instruction execution status and resource usage in real time, and to feed back interrupt or status register information to the processor unit.

[0066] For example, the processor unit writes data to the shared memory space through the data port and sends tensor instructions to the scheduling unit through the control interface. The scheduling unit performs dependency checks and sends micro-operation control signals. The general-purpose parallel neural network processor reads data directly from the specified address in the memory space for computation based on the micro-operation control signals sent by the scheduling unit, and writes the result directly back to the memory space. After the computation is completed, it notifies the scheduling unit of completion, and the scheduling unit updates the status and notifies the processor unit.

[0067] Overall, a new RVGPNPU architecture is formed by integrating a processor unit with a fifth-generation instruction set architecture (RISC-V) core and a general-purpose parallel neural network processor into a single IP core. Within the IP core, a hardware scheduling unit with dependency resolution and topology mapping functions is integrated. This unit directly resolves the dependencies of tensor instructions and generates micro-control signals that adapt to the boundaries and core characteristics of the improved neural network processor array, thereby enabling the decentralization of processor functions and fine-grained parallelism in heterogeneous computing.

[0068] The neural network acceleration unit is a heterogeneous accelerator that performs tensor operations. Its internal architecture includes a finite state machine (FSM) and a reconfigurable array of process elements (PEs). The FSM module is the global control center of the general-purpose parallel neural network processor, responsible for parsing micro-operation signals from the scheduling unit and controlling internal bus switching, data flow, and computation mode switching. The process element module is the computational carrier controlled by the FSM, including boundary processing units and array processing units. Boundary processing units are responsible for data I / O and format conversion, while array processing units are responsible for high-density matrix operations. They support dynamic reconfiguration into different computational topologies (such as a 4×4 global array or multiple 2×2 local arrays) through the FSM. The dependency check table is a high-speed register file or static random access memory (SRAM) table stored inside the scheduling unit.

[0069] In one exemplary embodiment, such as Figure 2 As shown, a task scheduling method is provided, applied to a scheduling unit, and the method includes:

[0070] Step S210: Receive tensor instructions distributed by the processor unit.

[0071] Tensor instructions are custom extension instructions that map computationally intensive operators (such as convolution and matrix multiplication) to the neural network model during compilation by the compiler or development toolchain. They are executed by dedicated computational units (including multiply-accumulate units and tensor register sets) of the general-purpose parallel neural network processor. These instructions directly relate to multi-dimensional tensor data, triggering multiple processing units within the general-purpose parallel neural network processor's computational array to simultaneously perform the same operation. This drives the general-purpose parallel neural network processor's computational array to perform large-scale, highly parallel tensor operations, such as matrix multiplication, convolution, and vector dot products.

[0072] For example, when executing the instruction stream, the processor unit performs instruction pre-decoding. When the instruction is identified as a custom tensor extension instruction, the processor unit does not execute it but instead routes the instruction to the scheduling unit's instruction queue via hardware bypass. At this time, the processor unit can immediately continue executing subsequent unrelated scalar instructions, achieving non-blocking operation. Scalar instructions are the basic general-purpose instructions of the RISC-V core, used to handle single-value operations and control logic, covering functions such as arithmetic operations, logical judgments, data movement, and system control.

[0073] It should be understood that the processor unit can also write tensor instructions to a fixed physical address of the scheduling unit via load instructions. The scheduling unit hardware monitors this set of registers in real time, and once the writing is complete and the flag bit is 1, it immediately reads and clears the flag bit. Through at least two of the above hardware direct connection methods, zero software overhead, high bandwidth, and low latency tensor instruction distribution from the processor unit to the scheduling unit is achieved. This step achieves preliminary decoupling of control flow and computation flow, breaking the bottleneck of the processor needing to block and wait for the response of the neural network acceleration unit in the traditional architecture, and significantly improving the processor utilization and the system's concurrent processing capability.

[0074] Step S220: Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information.

[0075] The dependency check table is a hardware storage structure configured within the scheduling unit. It maintains the current task state context of the system, recording the real-time occupancy of critical resources and the relevant attributes of the instructions being executed. The hardware implementation of this dependency check table can be a fully hardware content-addressable memory (WRAM), with entries using tensor instruction numbers as keys, and each entry storing dependency check information tags in parallel. Alternatively, the dependency check table can be physically split into multiple independent hardware tables, each storing the corresponding dependency check information tags.

[0076] Dependency check information refers to the status data read from the dependency check table used to determine whether the current instruction meets the conditions for execution. This data reflects the logical order requirements between instructions and the availability of hardware resources. Dependency checking is the process by which the scheduling unit uses hardware logic circuits to compare the requirements of the tensor instruction to be scheduled with the dependency check information to determine whether there are any conflicts that would prevent instruction execution.

[0077] For example, the scheduling unit first parses the received tensor instruction and extracts the key feature parameters required for the instruction's execution, such as the required source operand index, the target hardware resource identifier, and the data addresses involved. Subsequently, the scheduling unit uses these feature parameters as indexes to access the internal dependency check table, read the current system state related to the instruction, and obtain dependency check information.

[0078] The hazard detection logic circuit inside the scheduling unit receives the aforementioned dependency check information and logically compares it with the feature parameters of the tensor instruction. It confirms whether the input data required by the instruction has already been prepared by the preceding instruction, i.e., there is no data dependency conflict; and confirms whether the hardware resources inside the neural network acceleration unit required by the instruction are currently idle, i.e., there is no resource contention conflict.

[0079] Based on the comparison results, the scheduling unit generates a pass / fail signal. If the check fails, the scheduling unit will postpone the issuance of the instruction; if the check passes, the subsequent instruction translation process will begin. This step utilizes a dedicated hardware dependency check table to replace traditional software scheduling queries. On the one hand, hardware table lookup significantly reduces scheduling latency, enabling rapid completion of complex dependency judgments in a pipelined manner. On the other hand, by maintaining dependency check information, the system can accurately identify data hazards and resource conflicts, ensuring data consistency and execution correctness in out-of-order issuance or parallel execution scenarios, and avoiding computational errors or deadlocks caused by blind issuance.

[0080] In step S230, if the dependency check passes, the tensor instruction is converted into a micro-operation control signal and the micro-operation control signal is sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0081] The implementation of converting tensor instructions into micro-operation control signals varies depending on the accelerator. It should be understood that when the scheduling unit integrates a hardwired microcode read-only memory, it can directly look up a table based on the tensor instructions to output a fixed or variable-length sequence of micro-operation control signals; when the scheduling unit has a built-in programmable microcode processor, it can dynamically execute microcode programs to generate micro-operation control signals; and when the scheduling unit has a multi-stage pipelined decoder, it can generate micro-operation control signals in stages.

[0082] For example, when all dependency checks pass, the scheduling unit's decoding logic converts the tensor instruction into micro-operation control signals recognizable within the neural network acceleration unit. The scheduling unit updates the dependency check table, marking the processing unit resources involved in the instruction as locked and the relevant memory address range as occupied. The scheduling unit sends the micro-operation control signals to the finite state machine module of the neural network acceleration unit. The finite state machine parses the signals, controls the vertical bus and horizontal shift register, reassembles multiple processing unit modules into the target topology, and directs the processing units to directly read data from the shared memory space via physical address pointers for computation.

[0083] In this step, the scheduling unit converts macroscopic tensor instructions into low-level micro-operation signals, shielding the complexity of the neural network acceleration unit and reducing the decoding burden on the processor. Secondly, by directly commanding the finite state machine within the neural network acceleration unit, fine-grained scheduling of the low-level processing units is achieved. This allows for flexible adaptation to different computing topologies, maximizing the utilization and flexibility of the neural network acceleration unit's hardware resources while ensuring scheduling efficiency.

[0084] Step S240: Receive the execution completion signal fed back by the neural network acceleration unit after completing the tensor operation, update the dependency check table, and send an operation notification message to the processor unit.

[0085] For example, when the general-purpose parallel neural network processor completes computation or encounters an overflow error, the finite state machine sends an execution completion signal to the scheduling unit via a dedicated signal line. In response to this signal, the scheduling unit immediately operates the dependency check table, clearing the corresponding resource occupancy flags and address range locks, thereby releasing resources for subsequent instructions in the waiting queue. Finally, the scheduling unit sends a notification message to the processor unit. This notification message can be written back to the processor unit's status register or sent as a lightweight hardware interrupt to inform the processor unit that the tensor instruction has been executed and post-processing of the result data is possible.

[0086] Automatically updating the dependency checklist ensures that subsequent suspended instructions can be immediately activated and executed, reducing pipeline idling time. Proactively sending notifications to the processor eliminates the need for inefficient polling, allowing the processor to operate in low-power mode or handle other tasks until the results are actually needed, further reducing overall system power consumption.

[0087] In one embodiment, the processor unit and the neural network acceleration unit share a common memory space and use a unified memory address encoding.

[0088] Tensor instructions carry memory address codes that point to tensor data in shared memory.

[0089] Converting tensor instructions into micro-operation control signals includes converting tensor instructions into micro-operation control signals containing memory address codes, so as to instruct the neural network acceleration unit to directly read tensor data from the shared memory space for computation according to the memory address codes.

[0090] For example, the processor unit and the neural network acceleration unit are connected to the same shared memory space, where all data uses a unified address encoding. The processor unit reads and issues tensor instructions, with the memory address encoding carried in the instructions precisely pointing to the tensor data to be processed in the shared memory. The scheduling unit parses the tensor instructions into micro-operation control signals, completely preserving the address encoding information. The micro-operation control signals are sent to the finite state machine module. After parsing the address encoding, the finite state machine instructs the neural network acceleration unit to directly read the tensor data from the corresponding address in the shared memory space, without the need for the processor unit as an intermediary, and directly execute tensor operations.

[0091] In one embodiment, the dependency check information for tensor instructions includes scalar tensor association markers and memory address occupancy markers. The scalar tensor association markers are used to characterize the association between scalar instructions and tensor instructions. The memory address occupancy markers are used to characterize the range of memory addresses that the processor unit or neural network acceleration unit is operating on.

[0092] Dependency checks are performed based on dependency check information, including suspending tensor instructions in the hardware queue of the scheduling unit if a dependency check fails, until the dependency check passes.

[0093] To support the accurate dependency checking in step S220, this embodiment, during the dependency checking process, after the scheduling unit receives the tensor instruction, it performs hazard detection checks in parallel. If any hazard exists, the instruction is suspended directly, without waiting for the detection results from other modules, thus avoiding delays caused by serial detection. Hazard detection includes the following two key fields:

[0094] The scalar-tensor association flag is used to characterize the association between scalar and tensor instructions. To solve synchronization problems in heterogeneous computing, it records the scalar register number that the current tensor instruction depends on. The memory address occupancy flag is used to characterize the range of memory addresses that the processor unit or neural network acceleration unit is operating on. To resolve read-write conflicts in shared memory, it records the physical address range information of the data read or written by the instruction in the shared memory space, specifically including the starting physical address, data length, and access step size.

[0095] like Figure 3 As shown, the dependency check step in step S220 includes the following steps:

[0096] In step S310, the scheduling unit receives the tensor instruction, queries the dependency check table, and obtains the dependency check information of the tensor instruction.

[0097] Step S320: Initiate hazard detection in parallel.

[0098] Step S331, Heterogeneous data hazard detection, determines whether there are unready dependencies between scalar operands and tensor operands.

[0099] In one embodiment, the dependency check information includes a scalar tensor association flag. The dependency check includes determining, based on the scalar tensor association flag, whether the operands of a tensor instruction depend on a scalar instruction that the processor unit has not yet completed. If so, a heterogeneous data hazard is identified, and the dependency check fails.

[0100] For example, when the scheduler detects that a tensor instruction to be scheduled requires scalar data from a processor unit, a heterogeneous data hazard check is triggered. The scheduler parses the source operand index from the instruction to be scheduled, identifying that the operand points to a scalar register in the processor unit. The scheduler queries the scalar tensor association flags in the dependency check table to check if there is a preceding instruction that has not yet completed writing to that scalar register. If a matching write operation flag is found, a heterogeneous data hazard is determined to exist. At this time, the scheduler sets the blocking signal for the instruction, suspending it to the hardware wait queue until the preceding instruction completes its write-back and clears the flag.

[0101] This embodiment addresses the synchronization issues unique to heterogeneous computing architectures by effectively preventing tensor operations from preemptively executing before the processor unit has completed calculating the required scalar parameters (such as dynamic step size and quantization coefficients). This avoids calculation errors and ensures the logical correctness of the program. Only instructions with explicit dependencies are intercepted; instructions without dependencies can still be processed by the scheduling unit, avoiding the performance loss caused by using coarse-grained global barriers for data security.

[0102] Step S332, address overlap hazard detection, determines whether the tensor input / output address overlaps with the addressing range of other instructions.

[0103] In one embodiment, the dependency check information includes storage address occupancy markers. The dependency check includes: calculating the target address range involved in the tensor instruction; and determining whether the target address range intersects with the address range of incomplete instructions recorded in the dependency check table. If there is an intersection and a read / write conflict exists, an address overlap hazard is identified, and the dependency check fails.

[0104] For example, to achieve efficient zero-copy parallelism in a shared memory architecture, the scheduling unit performs address overlap hazard checks. The arithmetic logic unit within the scheduling unit calculates the coverage area of ​​the target data in the storage space based on the parameters of the instruction to be scheduled, i.e., generates a target address range ([Start_A, End_A]). This target address range is then compared in parallel with the address range markers ([Start_B, End_B]) of all incomplete instruction records in the dependency check table. If an intersection is found, the operation attributes are further checked. A conflict is considered only if at least one of them is a write operation (i.e., RAW / WAR / WAW). If an intersection exists and the read / write attributes conflict, an address overlap hazard is determined, and the instruction is suspended.

[0105] This embodiment accurately identifies read-write conflicts (such as read-after-write and write-after-read) during tensor data block operations, preventing data overwriting or reading dirty data due to out-of-order execution or pipeline overlap. By recording state information in three dimensions, it comprehensively covers common data hazards, structural hazards, and control hazards in heterogeneous computing, ensuring the safety of instruction execution.

[0106] In step S340, if any module detects an hazard, it suspends the current tensor instruction, stores it in the hardware task queue, and waits for the dependency to be resolved.

[0107] For example, when any hazard exists, there is no need to wait for the detection results of other modules. After suspension, the instruction is stored in the hardware task queue, and the scheduling unit will monitor the dependency check table in real time. When the hazard is resolved, that is, when the scalar parameters are ready and the addresses do not overlap, the detection will be automatically retried.

[0108] In step S350, all modules are safe and convert tensor instructions into micro-operation control signals, which are then sent to the computing array.

[0109] With data ready and address safe, the process directly enters the instruction translation stage, ensuring that tensor instructions are quickly delivered to the general-purpose parallel neural network processor for execution without additional latency. The entire process is automatically executed by the scheduling unit's hardware logic, eliminating the need for software intervention and polling, thus improving execution efficiency.

[0110] In one embodiment, the processing unit module includes a boundary processing unit and an array processing unit.

[0111] The micro-operation control signal is sent to the finite state machine module to cause the finite state machine module scheduling and processing unit module to perform tensor operations, including:

[0112] The micro-operation control signal is sent to the finite state machine module, instructing the finite state machine module to schedule the boundary processing unit to perform data format conversion or edge filling tasks.

[0113] And / or,

[0114] The micro-operation control signal is sent to the finite state machine module, which is instructed to control the bus connection state between the array processing units according to the micro-operation control signal, so as to reorganize multiple array processing units into the target computing array.

[0115] Specifically, the micro-operation control signals issued by the scheduling unit are first transmitted to the finite state machine module built into the general-purpose parallel neural network processor. The finite state machine intelligently schedules the boundary processing unit and / or array processing unit to perform tensor operations according to the task type and calculation parameters of the micro-operation instructions, ensuring that different types of neural network computing tasks (such as convolution, fully connected, and pooling) can be efficiently adapted.

[0116] For example, when the task type of the micro-operation control signal is convolution operation, and carries edge padding or data format conversion requirements, the finite state machine parses the micro-operation control signal and extracts key parameters, such as input data format (INT8), target data format (FP16), edge padding mode (mirror padding), and convolution kernel size (3×3). The finite state machine sends control instructions to the boundary processing unit, specifying the task type (data format conversion first, then edge padding). The boundary processing unit responds to the control instructions and reads the input tensor to be processed (such as a "3-channel × 224 × 224" INT8 format feature map) from shared memory. Through the built-in format conversion circuit, the INT8 type input tensor is converted to FP16 type element by element, and the converted data is temporarily stored in the internal buffer of the boundary processing unit. According to the convolution kernel size (3×3) and padding mode (mirror padding), the FP16 feature map after format conversion is edge-extended, and one row / column of pixels is filled on each of the top, bottom, left, and right sides of the feature map (mirror copying the original edge pixels), generating a "3-channel × 226 × 226" padded tensor. After the boundary processing unit completes the preprocessing, it transmits the filled FP16 tensor to the local register group of the array processing unit through the internal bus and feeds back the preprocessing completion signal to the finite state machine.

[0117] For example, when the task type of the micro-operation control signal is a fully connected operation, and the computational dimension parameters are an input vector dimension of 1024 and an output vector dimension of 256, the finite state machine parses the micro-operation control signal, extracts the target array topology parameters as a parallel split mode, and reassembles them into 8 groups of 4×1 processing unit sub-arrays. At this time, the finite state machine sends a configuration command to the bus control module of the array processing unit to control the bus connection state between the processing units: dividing the 32 processing units into 8 independent sub-arrays, each sub-array containing 4 processing units. The processing units within each sub-array achieve data interaction through serial cascading, and the sub-arrays are isolated through a parallel bus to avoid data interference. The array processing unit bus control module responds to the command, completes the topology reorganization of the processing unit array, forms 8 groups of 4×1 target computing arrays, and feeds back an array reorganization completion signal to the finite state machine.

[0118] The finite state machine sends computation instructions to the recombined array processing unit, instructing it to receive the preprocessed input tensor (1024-dimensional FP16 vector) and model weights (1024×256-dimensional FP16 matrix) from the boundary processing unit. Eight subarrays of processing units perform matrix multiplication in parallel: each subarray is responsible for multiplying the 32-dimensional input vector with its corresponding weight submatrix (32×256). Finally, the eight results are combined to generate a 256-dimensional output vector.

[0119] For example, when the task type of the micro-operation control signal is a convolution operation with format conversion, the finite state machine implements the coordinated scheduling of two processing units. The finite state machine first schedules the boundary processing unit to perform INT8 to FP16 format conversion and edge padding preprocessing. After the boundary processing unit completes the preprocessing, the finite state machine immediately schedules the array processing unit to reassemble into a 16×2 target computing array (adapting to the parallel granularity of the convolution operation). After the array processing unit is reassembled, the finite state machine sends the instruction to the array processing unit to receive the preprocessed data and execute the convolution operation. The boundary processing unit directly transmits the preprocessed tensor to the local register group of the array processing unit, and the array processing unit starts the 16×2 processing unit array to execute the convolution operation in parallel. During the operation, the finite state machine monitors the state of the two units in real time to ensure the synchronization of data transmission and operation execution.

[0120] This embodiment dynamically changes the topology between array processing units through micro-operation control signals, allowing the same hardware architecture to be flexibly reconfigured to adapt to different types of neural network layers (such as convolutional layers and fully connected layers) or different data flow patterns (such as systolic arrays and tree accumulation), thus improving the versatility of dedicated hardware. By setting up dedicated boundary processing units to handle auxiliary tasks such as padding or format conversion, the array processing units are freed from tedious data preprocessing, allowing them to focus on core multiply-accumulate operations, thereby improving the overall computing power density and energy efficiency.

[0121] In one embodiment, obtaining dependency check information for tensor instructions includes processing unit status flags, which characterize the occupancy status and topology of processing unit modules. To address the issue of space resource contention, the system records, in the form of a bitmap or a numbered list, which processing unit resource numbers the instruction exclusively occupies, and the current topology of these processing units.

[0122] Dependency checks are performed based on dependency check information, including:

[0123] The target processing unit resource number and target topology required to parse tensor instructions;

[0124] Query the status flag of the processing unit and compare the resource number of the target processing unit with the occupied resource number and the corresponding current topology configuration recorded in the status flag of the processing unit;

[0125] In cases of overlap, the dependency check will fail.

[0126] like Figure 4 As shown, the dependency check step in step S220 includes the following steps:

[0127] In step S410, the scheduling unit receives the tensor instruction, queries the dependency check table, and obtains the dependency check information of the tensor instruction.

[0128] Step S420: Initiate hazard detection in parallel.

[0129] Step S431, Heterogeneous data hazard detection, determines whether there are unready dependencies between scalar operands and tensor operands.

[0130] Step S432, address overlap hazard detection, determines whether the tensor input / output address overlaps with the addressing range of other instructions.

[0131] Step S433: Resource dependency hazard detection, determining whether the processing unit resources are occupied.

[0132] In one embodiment, the dependency check information includes a processing unit status flag. The dependency check includes: the target processing unit resource number and target topology required to parse the tensor instruction; and comparing the target processing unit resource number with the occupied resource number and corresponding current topology configuration recorded in the processing unit status flag. If there is overlap, a resource dependency hazard is determined to exist, and the dependency check fails.

[0133] For example, taking advantage of the reconfigurable nature of general-purpose parallel neural network processors, the scheduling unit performs resource dependency hazard checks to avoid multiple tasks competing for physical computing resources. The scheduling unit parses the control field of the tensor instruction to be scheduled, extracting the target processing unit resource number (e.g., requesting PE#4) and the required target topology (e.g., independent computing mode). This target information is compared with the processing unit resource occupancy status recorded in the dependency check table. It checks whether the target number (PE#4) is included in the recorded set of occupied resources. Even if the number is not directly occupied, if the hardware partition to which the processing unit belongs is in a global configuration mode incompatible with its target topology (e.g., the array block containing PE#4 is configured in global broadcast mode), it is considered a conflict. If any of these conflicts exist, a resource dependency hazard is determined, and the instruction is suspended.

[0134] This embodiment, by accurately comparing resource numbers and topology, allows different sub-modules within the neural network acceleration unit to execute different instructions in parallel without resource conflicts, thereby improving the hardware's concurrent processing capabilities. It effectively avoids hardware deadlocks or abnormal states caused by multiple instructions simultaneously contending for the same computing node or bus resources, ensuring the accelerator's operational stability in complex multi-tasking scenarios.

[0135] In step S440, if any module detects an hazard, it suspends the current tensor instruction, stores it in the hardware task queue, and waits for the dependency to be resolved.

[0136] For example, when any hazard exists, there is no need to wait for the detection results of other modules. After suspension, the instruction is stored in the hardware task queue, and the scheduling unit will monitor the dependency check table in real time. When the hazard is resolved, that is, when the scalar parameters are ready, the addresses do not overlap, and the resources are not occupied, the detection will be automatically retried.

[0137] In step S450, all modules are safe and convert tensor instructions into micro-operation control signals, which are then sent to the computing array.

[0138] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0139] Based on the same inventive concept, this application also provides a task scheduling apparatus for implementing the task scheduling method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more task scheduling apparatus embodiments provided below can be found in the limitations of the task scheduling method described above, and will not be repeated here.

[0140] In one exemplary embodiment, such as Figure 5 As shown, a task scheduling device 500 is provided, integrated into a scheduling unit. The scheduling unit is connected to a processor unit and a neural network acceleration unit. The neural network acceleration unit includes a finite state machine module and multiple processing unit modules controlled by the finite state machine module. The device includes:

[0141] The instruction receiving module 510 is used to receive tensor instructions distributed by the processor unit and send the tensor instructions to the dependency checking module 520.

[0142] The dependency checking module 520, connected to the instruction receiving module 510, is used to query the dependency checking table, obtain the dependency checking information of the tensor instruction, perform dependency checks based on the dependency checking information, and send the tensor instruction to the instruction distribution module 530 if the dependency check passes.

[0143] The instruction dispatch module 530, connected to the dependency checking module 520, is used to convert tensor instructions into micro-operation control signals and send the micro-operation control signals to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations.

[0144] The status feedback module 540, connected to the dependency checking module 520, is used to receive the execution completion signal fed back by the neural network acceleration unit after completing the tensor operation, update the dependency checking table, and send the operation notification message to the processor unit.

[0145] In one embodiment, the processor unit and the neural network acceleration unit share a common memory space and use a unified memory address encoding.

[0146] The tensor instructions in the instruction receiving module 510 carry memory address encoding, which points to tensor data in the shared memory space.

[0147] The instruction dispatch module 530 converts tensor instructions into micro-operation control signals, including:

[0148] Tensor instructions are converted into micro-operation control signals containing memory address codes, instructing the neural network acceleration unit to directly read tensor data from the shared memory space for computation based on the memory address codes.

[0149] In one embodiment, the dependency checking module 520 obtains dependency checking information for tensor instructions, including scalar tensor association markers and memory address occupancy markers. The tensor association markers are used to characterize the association relationship between scalar instructions and tensor instructions; the memory address occupancy markers are used to characterize the range of memory addresses that the processor unit or neural network acceleration unit is operating on.

[0150] Dependency checks are performed based on dependency check information, including suspending tensor instructions in the hardware queue of the scheduling unit if a dependency check fails, until the dependency check passes.

[0151] In one embodiment, the dependency checking module 520 performs dependency checks based on dependency checking information, including:

[0152] Query the scalar tensor association flag to determine whether the operands of a tensor instruction depend on a scalar instruction that has not yet been completed by the processor unit.

[0153] Parse the target read / write address range involved in the memory address encoding in the tensor instruction; query the memory address occupancy flag to determine whether the target read / write address range is locked by an incomplete write operation of the processor unit.

[0154] If there are dependencies or the system is locked, the dependency check will fail.

[0155] In one embodiment, the processing unit module includes a boundary processing unit and an array processing unit.

[0156] The instruction dispatch module 530 sends micro-operation control signals to the finite state machine module, so that the finite state machine module scheduling and processing unit module performs tensor operations, including:

[0157] The finite state machine module is instructed to schedule the boundary processing unit to perform data format conversion or edge filling tasks.

[0158] And / or,

[0159] The finite state machine module is instructed to control the topology between array processing units according to micro-operation control signals, so as to reorganize multiple array processing units into a target computing array.

[0160] In one embodiment, the dependency checking module 520 obtains dependency checking information for tensor instructions, which also includes a processing unit status flag to characterize the occupancy status and topology of the processing unit module.

[0161] Dependency checks are performed based on dependency check information, including:

[0162] The target processing unit resource number and target topology required to parse tensor instructions.

[0163] The query processing unit status flag is used to compare the target processing unit resource number with the occupied resource number and the corresponding current topology configuration recorded in the processing unit status flag.

[0164] In cases of overlap, the dependency check will fail.

[0165] Each module in the aforementioned task scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0166] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores task scheduling data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. The computer program is executed by the processor to implement a task scheduling method.

[0167] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0169] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0170] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0171] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0172] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0173] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0174] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task scheduling method, characterized in that, The method is applied to a scheduling unit, which is connected to a processor unit and a neural network acceleration unit, respectively. The neural network acceleration unit includes a finite state machine module and multiple processing unit modules controlled by the finite state machine module. Receive tensor instructions distributed by the processor unit; Query the dependency check table to obtain the dependency check information of the tensor instruction, and perform dependency checks based on the dependency check information; If the dependency check passes, the tensor instruction is converted into a micro-operation control signal and sent to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations. The system receives the execution completion signal fed back by the neural network acceleration unit after completing the tensor operation, updates the dependency check table, and sends an operation notification message to the processor unit.

2. The method according to claim 1, characterized in that, The processor unit and the neural network acceleration unit share a common memory space and use a unified memory address encoding. The tensor instruction carries the memory address encoding, which points to the tensor data in the shared memory space; The step of converting the tensor instruction into a micro-operation control signal includes: The tensor instructions are converted into micro-operation control signals containing the memory address encoding, instructing the neural network acceleration unit to read tensor data from the shared memory space for computation according to the memory address encoding.

3. The method according to claim 2, characterized in that, The dependency check information of the tensor instruction includes a scalar tensor association flag and a memory address occupancy flag, wherein the scalar tensor association flag is used to characterize the association relationship between the scalar instruction and the tensor instruction; the memory address occupancy flag is used to characterize the range of memory addresses that the processor unit or the neural network acceleration unit is operating on. The dependency check based on the dependency check information includes suspending the tensor instruction in the hardware queue of the scheduling unit if the dependency check fails, until the dependency check passes.

4. The method according to claim 3, characterized in that, The dependency check based on the dependency check information includes: Query the scalar tensor association flag to determine whether the operands of the tensor instruction depend on the scalar instruction that the processor unit has not yet completed; The target read / write address range involved in the memory address encoding in the tensor instruction is parsed. Query the memory address occupancy flag to determine whether the target read / write address range is locked by the incomplete write operation of the processor unit; The dependency check fails if there are dependencies or if the device is locked.

5. The method according to claim 1, characterized in that, The processing unit module includes a boundary processing unit and an array processing unit; Sending the micro-operation control signal to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations includes: The micro-operation control signal is sent to the finite state machine module, instructing the finite state machine module to schedule the boundary processing unit to perform data format conversion or edge filling tasks; And / or, The finite state machine module is instructed to control the topology between the array processing units according to the micro-operation control signal, so as to reorganize multiple array processing units into a target computing array.

6. The method according to claim 1, characterized in that, The dependency check information includes a processing unit status flag, which is used to characterize the occupancy status and topology of the processing unit module. The dependency check based on the dependency check information includes: The target processing unit resource number and target topology required to parse the tensor instruction; Query the status flag of the processing unit, and compare the resource number of the target processing unit with the occupied resource number and the corresponding current topology configuration recorded in the status flag of the processing unit; The dependency check fails if there is overlap.

7. A task scheduling device, characterized in that, Integrated into a scheduling unit, the scheduling unit is connected to a processor unit and a neural network acceleration unit, respectively. The neural network acceleration unit includes a finite state machine module and multiple processing unit modules controlled by the finite state machine module. The device includes: The instruction receiving module is used to receive tensor instructions distributed by the processor unit; The dependency checking module is used to query the dependency checking table, obtain the dependency checking information of the tensor instruction, and perform dependency checks based on the dependency checking information. The instruction dispatch module is used to convert the tensor instruction into a micro-operation control signal when the dependency check passes, and send the micro-operation control signal to the finite state machine module so that the finite state machine module schedules the processing unit module to perform tensor operations. The status feedback module is used to receive the execution completion signal fed back by the neural network acceleration unit after completing the tensor operation, update the dependency check table, and send an operation notification message to the processor unit.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images