Fine-grained multi-operator parallel scheduling method and system based on heterogeneous data stream architecture

Abstract

The invention relates to a fine-grained multi-operator parallel scheduling method and system based on a heterogeneous data stream architecture. The method comprises the steps that an operator topological graph is established according to operators obtained through decomposition and related to the task scheduling process, and an operator data flow graph is constructed; analyzing a data dependency relationship among operators in a task according to the operator topological graph, grouping the operators based on the data dependency relationship and the runtime characteristics of the operators, and maintaining a corresponding scheduling queue for each operator group; scheduling the parallel operator group according to the scheduling queue, and allocating computing resources for the parallel operators based on runtime characteristics; performing data flow diagram dynamic reconstruction on the original operator data flow diagram of the parallel operator group according to the operator data flow diagram; and after the PE data flow array finishes executing the operators in parallel, checking whether each scheduling queue is empty or not, if so, ending operator scheduling, and otherwise, continuing to schedule the parallel operator group according to the scheduling queues. According to the method, the access to the SPM can be reduced, and multi-operator efficient parallel scheduling is realized.

Description

Technical Field

[0001] The present invention relates to the field of data flow technology, and in particular to a fine-grained multi-operator parallel scheduling method and system based on a heterogeneous data flow architecture. Background Art

[0002] In the field of modern computing, technology is gradually shifting from general computing to special computing, and the integration of big data, graph computing, and artificial intelligence has become a mainstream trend. This kind of fusion computing is characterized by data density, complex control and storage access, and poses challenges to computer systems with a single data flow or control flow architecture. An important issue is that in multiple computing modes, the mixed execution efficiency of operators is not high, which is manifested in fixed scheduling strategies, single execution order, and under-utilization of parallelism between operators.

[0003] The dataflow architecture creates a dataflow graph between processing units through a data-driven mechanism, aiming to reduce storage access and synchronization overhead. This architecture performs well in the fields of deep learning and scientific computing because it can effectively improve data parallelism and resource utilization. The concept of dataflow originated from dataflow computers and has evolved from instruction-level parallelism to thread-level parallelism. Representative examples include the Codelet model and the EARTH dataflow computer.

[0004] In the execution of data flow programs, a computing task is decomposed into multiple small tasks, which are organized into a data flow graph (directed acyclic graph), where nodes represent computing tasks and edges represent task dependencies. In the current implementation of data flow architecture, the task scheduling capability is greatly limited. The system generally executes tasks sequentially and lacks the ability to dynamically adjust the order.

[0005] In order to solve these problems, the prior art provides a new heterogeneous data flow architecture DFU. This architecture is built based on the RISCV core and the PE data flow array, and includes: the PE data flow array for data flow calculation, the RISCV core for controlling array operations, the DFU_Host main memory responsible for data storage, the SPM for data cache, and the register stack for storing operands and opcodes. This architecture creates a data flow graph for the operator and deploys it on the PE data flow array. The input data is transferred from the DRAM of the RISCV core to the shared SPM via DMA. The PE data flow array extracts data from the SPM for processing, achieving high parallelism and significantly reducing memory access time.

[0006] However, the current PE dataflow array only supports the dataflow execution of a single operator, and lacks corresponding scheduling strategies and runtime support for fine-grained parallelism of multiple operators. In a multi-tasking environment, tasks can be divided into multiple different operators, but existing dataflow accelerators cannot effectively select and schedule appropriate operators for parallel execution. When multiple operators are executed in parallel, in order to improve data reuse and reduce memory access times, it is necessary to build a suitable dataflow graph for deployment. However, how to build a new dataflow graph from a single operator environment to support parallel operators is a challenge.

[0007] Currently, fine-grained multi-operator parallelism can be achieved by analyzing the data flow relationship between different operators and manually constructing a data flow graph. However, in a multi-task environment, the number of combinations of different operators is huge, and it is almost impossible to manually configure all fine-grained multi-operator parallel data flow graphs. Therefore, how to design a fine-grained multi-operator parallel scheduling method and runtime system on a new heterogeneous data flow architecture to dynamically select appropriate operators for parallel execution and dynamically generate appropriate data flow graphs is an urgent problem to be solved.

[0008] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology. Summary of the invention

[0009] The task scheduling capability of the current data flow architecture is limited. The system usually executes tasks sequentially and lacks the ability to dynamically adjust the order. To address this defect, the prior art proposes a new heterogeneous data flow architecture DFU (Device Firmware Upgrade). The architecture is built based on the RISCV core and the PE data flow array, including the PE data flow array, the RISCV core, the DFU_Host main memory, the SPM and the register stack. The architecture creates a data flow graph for the operator and deploys it on the PE data flow array. The input data is transferred from the DRAM of the RISCV core to the shared SPM via DMA. The PE data flow array extracts data from the SPM for processing, thereby achieving high parallelism and greatly reducing the memory access time. However, the current PE data flow array only supports the data flow execution of a single operator, and lacks corresponding scheduling strategies and runtime support for fine-grained parallelism of multiple operators. In a multi-tasking environment, tasks can be divided into multiple different operators, but the existing data flow accelerator cannot effectively select and schedule appropriate operators for parallel execution. When multiple operators are executed in parallel, in order to improve data reuse and reduce the number of memory accesses, it is necessary to build a suitable data flow graph for deployment. However, how to construct a new data flow graph from a single operator environment to support parallel operators is an unresolved problem.

[0010] Currently, fine-grained multi-operator parallelism can be achieved by analyzing the data flow relationship between different operators and manually constructing a data flow graph. However, in a multi-task environment, the number of combinations of different operators is huge, and it is almost impossible to manually configure all fine-grained multi-operator parallel data flow graphs. Therefore, how to design a fine-grained multi-operator parallel scheduling method and runtime system on a new heterogeneous data flow architecture to dynamically select appropriate operators for parallel execution and dynamically generate appropriate data flow graphs is an urgent problem to be solved.

[0011] In view of the deficiencies of the prior art, the present invention provides a fine-grained multi-operator parallel scheduling method based on a heterogeneous data flow architecture from a first aspect, the method comprising: establishing an operator topology graph according to the operators related to the task scheduling process obtained by decomposition, and constructing an operator data flow graph; analyzing the data dependencies between operators in the task according to the operator topology graph, grouping the operators based on the data dependencies and the runtime characteristics of the operators, and maintaining corresponding scheduling queues for each operator group; scheduling the parallel operator groups according to the scheduling queues, and allocating computing resources to the parallel operators to be parallelized in the parallel group based on the runtime characteristics; dynamically reconstructing the original operator data flow graph of the parallel operator group according to the operator data flow graph; when the PE data flow array completes the parallel execution of the operator, checking whether each scheduling queue is empty, if so, terminating the operator scheduling, otherwise, continuing to schedule the parallel operator groups according to the scheduling queues.

[0012] The present invention combines the operators to be scheduled into three different types of operator groups by analyzing the data dependencies between operators. According to the categories of the parallel scheduled operators, the subtask data flow graph of multiple operators in parallel is dynamically reconstructed to reduce the number of operator accesses to DRAM. The access to SPM is reduced through data flow between PEs, thus realizing efficient parallel scheduling of multiple operators.

[0013] The present invention combines the structure of the PE data stream array on the DFU accelerator to dynamically allocate the PE data stream array to multiple operators in the operator group, thereby achieving load balancing of PE computing resources, improving PE utilization, ensuring efficient parallelization of multiple operators, and reducing the idle time of computing resources.

[0014] According to a preferred embodiment, the method for constructing an operator data flow graph on a PE data flow array includes: dividing the decomposed operator into code segments; generating subtasks corresponding to each code segment; and the PE data flow array sequentially executing the subtasks. In this way, the fine-grained decomposition of the operator can be automatically performed and mapped to the PE data flow array for execution, which simplifies the complexity of manually constructing the data flow graph, enables the system to flexibly cope with a large number of operator combinations in a multi-task environment, and improves execution efficiency and system scalability.

[0015] According to a preferred embodiment, the method for grouping operators based on data dependency and runtime characteristics of operators includes: dividing operators into operator groups with data dependency, operator groups processing the same data, and operator groups processing different data; wherein, in the operator topology graph, identifying the dependency between each operator and its successor operator, if the operator has only one successor operator in the operator topology graph, combining the operator and the successor operator into a data-dependent operator group, and merging the operator with the successor operator if the capacity and number of registers of the SPM are sufficient. In this way, the dependency between operators can be effectively organized and managed, data transmission overhead can be reduced, and the parallelism and resource utilization of operator execution can be improved, thereby improving the overall computing performance.

[0016] According to a preferred embodiment, the method for grouping operators based on data dependencies and runtime characteristics of operators also includes: if an operator has multiple successor operators in the operator topology graph, performing at least one storage operation on the result data generated by the current operator, and combining all the successor operators into a data sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator. This reduces repeated calculation and transmission of data, optimizes the efficiency of system resource utilization, reduces the waste of memory and computing resources, and improves the overall execution speed and efficiency of the system.

[0017] According to a preferred embodiment, the method for grouping operators based on data dependency and runtime characteristics of operators also includes: when selecting operators in the data sharing operator group, based on a greedy method, dynamically updating the head pointer and tail pointer of the ordered list of data sharing operators, keeping the average PE utilization of the operator group close to the average PE utilization of the current operator group in the ordered list of data sharing operators, so as to avoid the PE utilization of the parallel operator group being too low. This optimizes the use efficiency of PE and ensures that the system can still run efficiently under high load.

[0018] According to a preferred embodiment, the method for allocating computing resources for parallel operators includes: calculating the computing resources of each operator according to the proportion of the occupancy rate of the CAL functional components of each operator scheduled in parallel; and allocating a corresponding number of processing units to the operators in the operator group according to the calculated computing resources. In this way, computing resources can be reasonably allocated, the number of processing units can be dynamically adjusted according to the actual needs of the operators, resource waste can be avoided, and the overall computing efficiency and resource utilization of the system can be improved.

[0019] From a second aspect, the present invention provides a fine-grained multi-operator parallel scheduling system based on a heterogeneous data flow architecture, including a processor, which is configured to: establish an operator topology graph according to the operators related to the task scheduling process obtained by decomposition, and construct an operator data flow graph; analyze the data dependencies between operators in the task according to the operator topology graph, group the operators based on the data dependencies and the runtime characteristics of the operators, and maintain corresponding scheduling queues for each operator group; schedule the parallel operator groups according to the scheduling queues, and allocate computing resources to the parallel operators to be parallel in the parallel group based on the runtime characteristics; dynamically reconstruct the original operator data flow graph of the parallel operator group according to the operator data flow graph; when the parallel execution of the operator ends, check whether each scheduling queue is empty, if so, end the operator scheduling, otherwise, continue to schedule the parallel operator group according to the scheduling queue.

[0020] By automatically establishing the operator topology and constructing the operator data flow graph through the processor, the data dependencies between operators can be accurately analyzed, and operators can be grouped and scheduled based on these dependencies and runtime characteristics. This efficient scheduling mechanism can make full use of hardware resources, especially the parallel computing capabilities of the processing unit (PE), dynamically adjust computing resource allocation and data flow graph reconstruction, improve the overall processing performance and resource utilization of the system, and reduce resource idleness and computing bottlenecks.

[0021] According to a preferred embodiment, the processor is also configured to: when executing the step of grouping operators based on data dependencies and runtime characteristics of operators, divide the operators into operator groups with data dependencies, operator groups that process the same data, and operator groups that process different data; wherein, in the operator topology graph, identify the dependency relationship between each operator and its successor operator, if the operator has only one successor operator in the operator topology graph, combine the operator and the successor operator into a data-dependent operator group, and if the capacity and number of registers of the SPM are sufficient, merge the operator with the successor operator.

[0022] When grouping operators, the processor can identify the dependencies in the operator topology graph and combine operators into different types of operator groups based on these dependencies. This grouping method optimizes the hardware resource allocation for operator execution by making full use of the SPM capacity and the number of registers, achieving optimal resource utilization, reducing data transmission overhead and waiting time, and improving the execution efficiency and response speed of the system.

[0023] According to a preferred embodiment, the processor is also configured to: when executing the step of grouping operators based on data dependencies and runtime characteristics of operators, if the operator has multiple successor operators in the operator topology graph, perform at least one storage operation on the result data generated by the current operator, and combine all successor operators into a data sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator.

[0024] When the processor processes an operator with multiple successor operators, it stores the result data and combines the successor operators into a data-sharing operator group. This method effectively avoids repeated calculations and repeated data loading, optimizes data storage and transmission paths, improves the utilization efficiency of storage resources and computing resources, reduces the waste of hardware resources and the overall processing time of the system, and improves the performance and efficiency of parallel computing.

[0025] According to a preferred embodiment, the processor is also configured to: in the step of grouping operators based on data dependencies and runtime characteristics of operators, when selecting operators in the data sharing operator group, dynamically update the head pointer and tail pointer of the ordered list of data sharing operators based on a greedy method to keep the average PE utilization of the operator group close to the average PE utilization of all operators in the ordered list of data sharing operators, so as to avoid too low PE utilization of the parallel operator group.

[0026] The processor uses a greedy algorithm to dynamically update the head pointer and tail pointer of the ordered list of data sharing operators, keeping the average PE utilization of the operator group close to the average PE utilization of all operators in the ordered list. Through this dynamic adjustment, the processor can evenly allocate hardware computing resources, avoid overload or idleness of some PEs, optimize the overall PE utilization, and ensure that the system can still run efficiently and stably under high load and complex task environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a fine-grained multi-operator parallel scheduling method based on a heterogeneous data flow architecture provided by the present invention;

[0028] Figure 2 It is a schematic diagram of the hardware structure of a data flow accelerator based on a heterogeneous data flow architecture provided by the present invention;

[0029] Figure 3 It is a schematic diagram of multi-operator hierarchical division of the heterogeneous data flow architecture provided by the present invention;

[0030] Figure 4 It is a schematic diagram of multi-operator grouping provided by the present invention;

[0031] Figure 5 It is a schematic diagram of a new data flow graph formed by splicing different operator subtasks provided by the present invention;

[0032] Figure 6 It is an operator topology diagram of the Unet network provided by the present invention;

[0033] Figure 7 It is a data flow diagram of the activation function Relu and the maximum pooling layer MaxPool in the Unet network provided by the present invention;

[0034] Figure 8 It is a schematic diagram of operator group classification of the Unet network provided by the present invention;

[0035] Fig. 9 It is a schematic diagram of reconstruction and scheduling of operators of four maximum pooling layers MaxPool for data sharing of the Unet network provided by the present invention. DETAILED DESCRIPTION

[0036] The following is a detailed description with reference to the accompanying drawings.

[0037] Heterogeneous dataflow architecture: A computing architecture that combines different types of processing units (such as general-purpose processors and special-purpose processors) to achieve efficient parallel computing through data flow graphs.

[0038] Multi-operator parallelism: A parallel computing method that allows multiple operations (or "operators") to be executed simultaneously to improve computing efficiency and resource utilization. In a heterogeneous data flow architecture, multi-operator parallelism can make full use of different types of computing resources to improve performance.

[0039] Task Scheduling: In computing tasks, task scheduling refers to the process of allocating computing resources (such as processing units) to perform tasks. Effective task scheduling algorithms can optimize resource utilization and improve computing efficiency and performance.

[0040] Operator topology graph: A graph structure that represents the data dependency relationship between multiple operators. Each node represents an operator, and the edge represents the data dependency relationship. By analyzing the operator topology graph, the execution order of tasks and the parallel opportunities can be determined.

[0041] PE data flow array: An array structure composed of multiple processing elements (PEs), which is specially used for data flow calculations. Each PE is connected by a data path, which can efficiently perform data flow tasks.

[0042] Operator data flow graph: In data flow computing, a graph structure that represents the execution order of operators and the relationship between data transfer. By constructing an operator data flow graph, operators can be efficiently scheduled and executed on the PE data flow array.

[0043] Data dependency: In a computing task, the output of a certain operator is used as the input of other operators, forming a data dependency.

[0044] Scheduling queue: During the task scheduling process, it is used to maintain the information queue of operators to be executed. According to different operator characteristics and data dependencies, scheduling queues can be divided into multiple types, such as data-dependent queues and data-independent queues.

[0045] Initial operator list: In the task scheduling process, it is the set of operators to be scheduled in the initial stage. The initial operator is usually an operator with an in-degree of 0 in the topology graph, that is, it does not rely on the input data of other operators.

[0046] SPM (Scratch-Pad Memory): A high-speed cache module used to temporarily store data used in the computation process. The size of SPM is an important factor affecting the efficiency of operator execution.

[0047] Average PE utilization: The average utilization of the processing element (PE), indicating the workload of the PE in a certain period of time. A high average PE utilization means that computing resources are fully utilized.

[0048] Dynamic reconstruction of data flow graph: During the calculation process, the data flow graph is adjusted and optimized according to the actual runtime situation. Dynamic reconstruction of data flow graph can improve computing efficiency and resource utilization and adapt to changing task requirements.

[0049] The meanings of the variables involved in the present invention are shown in Table 1.

[0050] Table 1: Meaning of each variable

[0051] variable Referential meaning <![CDATA[MEM i ]]> Peak memory usage when operator i processes a unit data block <![CDATA[OCC cali ]]> Occupancy rate of CAL functional components when operator i is running <![CDATA[UTL i ]]> PE utilization rate of operator i during runtime <![CDATA[REG i ]]> The number of registers required by operator i to process a unit data block when it is running list1 Starting operator list list2 Ordered list of data sharing operators MAXP Maximum number of parallel operators <![CDATA[Par r ]]> The number of parallel operators in the current operator group memSpm The total memory size of the operator group regNum The number of registers currently required by the operator group <![CDATA[ULT mean ]]> Average PE utilization of all operators in list2 <![CDATA[ULT cur ]]> Average PE utilization of the current operator group Resouce <![CDATA[Operator i Number of allocated PE computing resources]]> <![CDATA[OCC caltot ]]> Total CAL functional component occupancy rate of operator group <![CDATA[OCC i ]]> <![CDATA[Occupancy rate of the CAL functional component of the operator i >

[0052] Example 1

[0053] In the prior art, the current PE data flow array can only support the data flow execution of a single operator, and lacks corresponding scheduling strategies and runtime support for the fine-grained parallelism of multiple operators. For example, in a multi-task environment, tasks can be divided into multiple different operators, and it is impossible to select suitable operators for parallel execution on the current data flow accelerator; when multiple operators are executed in parallel, in order to improve data reuse and reduce the number of memory accesses, it is necessary to build a suitable data flow graph for deployment, and how to build a new data flow graph based on the data flow graph of the parallel operator in a single operator environment. At present, the purpose of fine-grained multi-operator parallelism can be achieved by analyzing the data flow relationship between different operators and manually constructing a data flow graph using these operators. However, in a multi-task environment, the large number of combinations of different operators makes it almost impossible to manually configure all fine-grained multi-operator parallel data flow graphs. Therefore, how to design a fine-grained multi-operator parallel scheduling method and runtime system on a new heterogeneous data flow architecture so that appropriate operators can be dynamically selected for parallel execution and appropriate data flow graphs can be dynamically generated is an urgent problem to be solved.

[0054] The present invention discloses a fine-grained multi-operator parallel scheduling system based on a heterogeneous data flow architecture, comprising a processor for executing a fine-grained multi-operator parallel scheduling method based on a heterogeneous data flow architecture.

[0055] Preferably, if Figure 2 As shown, the processor includes a Host with a RISC V core, DMA, DRAM, a memory controller, a SPM and a PE data stream array.

[0056] The host with RISC V core is connected to DMA through the data transfer port to realize data transfer between the host and the memory. DMA is connected to the storage controller through the memory data port to manage the data transfer between the memory and other components. The storage controller is connected to SPM through the cache port to control the read and write operations of data to ensure storage efficiency. SPM is connected to the PE data stream array through the data port to provide data storage support required for calculation. The storage controller is also connected to DMA to optimize data transfer management.

[0057] Host is the main control part of the processor, responsible for the execution of the operating system and application programs. RISC V core is an open source instruction set architecture (ISA) that instructs the processor how to execute programs.

[0058] Specifically, the Host is provided with a RISC V core, including a data transfer port. The DMA includes a memory data port, and manages the transfer of memory data through the port. The storage controller has a cache port for connecting with the SPM to optimize the data read and write speed. The SPM (data cache unit) includes a data port, which is dedicated to connecting with the PE data stream array to provide data storage support required for calculation.

[0059] The data transfer port is connected to the memory data port of the DMA, and the data transfer between the Host and the DMA is realized through the PCIe or AXI high-speed data bus. The DMA is connected to the storage controller through the memory data port, and uses the AXI bus or the same type of high-speed interface for data transmission. The storage controller is connected to the SPM through the cache port, and uses the DDR interface or LPDDR interface to ensure efficient data read and write operations. At the same time, the storage controller is also connected to the DMA through the AXI bus to optimize data transfer management. The data port of the SPM is connected to the PE data stream array through a dedicated high-speed data channel (such as AXI or a custom parallel bus) to ensure high-throughput data transmission and support the execution of complex computing tasks.

[0060] DMA (Direct Memory Access) allows different modules to read and write memory directly without going through the CPU. This improves data transfer efficiency and reduces the burden on the CPU.

[0061] DRAM (Dynamic Random-Access Memory) is a dynamic random access memory used to store data that needs to be accessed quickly. DRAM needs to be refreshed regularly to maintain data.

[0062] The storage controller manages the data read and write operations between the processor and the memory. It is responsible for arranging the data access sequence and optimizing memory performance.

[0063] SPM (Scratchpad Memory) is a small-capacity, high-speed memory used to temporarily store data, usually used to accelerate specific computing tasks and reduce access to main memory. In the present invention, SPM refers to a data cache shared by all PEs in a PE data stream array.

[0064] PE (Processing Element) data flow array: responsible for performing computationally intensive tasks. Each internal processing element (PE) can perform calculations independently, which makes data processing more efficient.

[0065] The CAL functional unit refers to a functional unit private to each PE and is used to perform computing operations.

[0066] In the PE data flow array, the time required for data flow between PEs is much shorter than the time it takes for PE to access the SPM. Data flow can be carried out between PEs and between functional components within PEs by executing the constructed data flow graph, thereby improving data reuse and reducing the number of accesses to the SPM cache.

[0067] like Figure 1 As shown, the fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture of the present invention includes the following steps, which are executed by the Host.

[0068] S1: Establish an operator topology graph based on the decomposed operators related to the task scheduling process, and construct an operator data flow graph.

[0069] S2: Analyze the data dependencies between operators in the task according to the operator topology graph, group the operators based on the data dependencies and the runtime characteristics of the operators, and maintain the corresponding scheduling queue for each operator group.

[0070] S3: Schedule the parallel operator group according to the scheduling queue, and allocate computing resources to the parallel operators to be run in parallel in the parallel group based on the runtime characteristics.

[0071] S4: Dynamically reconstruct the original operator data flow graph of the parallel operator group according to the operator data flow graph.

[0072] S5: After the PE data stream array completes the parallel execution of the operator, check whether all scheduling queues are empty. If so, terminate the operator scheduling. Otherwise, continue to schedule the parallel operator group according to the scheduling queue.

[0073] The present invention combines the operators to be scheduled into three different types of operator groups by analyzing the data dependencies between operators. According to the categories of the parallel scheduled operators, the subtask data flow graph of multiple operators in parallel is dynamically reconstructed to reduce the number of operator accesses to DRAM. The access to SPM is reduced through data flow between PEs, thus realizing efficient parallel scheduling of multiple operators.

[0074] The present invention combines the structure of the PE data stream array on the DFU accelerator to dynamically allocate the PE data stream array to multiple operators in the operator group, thereby achieving load balancing of PE computing resources, improving PE utilization, ensuring efficient parallelization of multiple operators, and reducing the idle time of computing resources.

[0075] When performing a series of operations in step S1, the in-degree indeg and out-degree outdeg of the operators in the operator topology graph must be maintained to facilitate subsequent operator group classification.

[0076] The steps of S1 are as follows.

[0077] S11: The operator topology map will be obtained in advance through breadth-first traversal (BFS) analysis on the RISCV core of the Host and stored in DRAM, and will not be transmitted to the SPM of the PE data flow array.

[0078] S12: The method for constructing an operator data flow graph on a PE data flow array includes: dividing the decomposed operators into code segments; generating subtasks corresponding to each code segment; and the PE data flow array executes the subtasks in sequence.

[0079] The data flow graph refers to the operations performed by each PE during the execution of the operator on the PE data flow array, as well as the data flow relationship between each PE.

[0080] Operators on the PE data flow array are executed by deploying data flow graphs on the PE data flow array. Due to hardware limitations, the operator execution process needs to be divided into multiple data flow graphs, each of which is called a subtask. In order to deploy operators on the PE data flow array, the operators need to be divided into code segments. For different code segments, different subtasks are generated and deployed on the PE data flow array. Subtasks are executed sequentially, such as Figure 3 shown.

[0081] like Figure 3 As shown, the total operator load includes the operators Subkernel0 and Subkernel1 executed sequentially. Operator Subkernel0 is divided into Task0, Task1, Task2, and Task3. These tasks Task0, Task1, Task2, and Task3 are scheduled in parallel to form an operator group. The operator group includes Task0, Task1, and Task2. The operator group is constructed into a data flow graph and deployed on the PE data flow array.

[0082] Operator Subkernel1 is divided into Task0, Task1, Task2 and Task3. Task0, Task1, Task2 and Task3 are scheduled in parallel to form two independent parallel operator groups. Each operator group includes Task0, Task1 and Task2. In each operator group, tasks Task0, Task1 and Task2 are executed in sequence. Parallel operator groups Task0, Task1 and Task2 are deployed on the PE data flow array to ensure efficient execution of tasks and optimal utilization of resources.

[0083] This can automatically decompose operators into fine-grained parts and map them to the PE data flow array for execution, simplifying the complexity of manually constructing data flow graphs, enabling the system to flexibly cope with a large number of operator combinations in a multi-tasking environment, and improving execution efficiency and system scalability.

[0084] The steps of S2 are as follows.

[0085] S21: Divide the operators into operator groups with data dependencies, operator groups processing the same data, and operator groups processing different data.

[0086] The Host in the data flow accelerator traverses all the generated operator topology graphs and extracts the operators with in-degree 0 in the operator topology graph. If the out-degree of the operator is 0, it is added to the data-independent queue PQ3, otherwise, it is added to the starting operator list list1.

[0087] The host in the data flow accelerator analyzes the runtime characteristics of different operators. The peak memory usage (MEM i ), the occupancy rate of the CAL functional components when operator i is running (OCC cali ), PE utilization rate of operator i during runtime (UTL i ) and the number of registers required by each operator i to process a unit data block when it is running (REG i ).

[0088] The various runtime characteristics of the operator can be obtained through the PE data flow array feedback data in the data flow accelerator. The total memory occupied by the parallel scheduled operator group and the number of registers currently required by the operator group are:

[0089] memSpm=MEM1+MEM2+…+MEM Parr .

[0090] regNum=REG1+REG2+…+REG Parr .

[0091] Among them, Parr represents the number of parallel operators in the current operator group, memSpm represents the total memory occupied by the operator group, regNum represents the number of registers currently required by the operator group, and MEM i Indicates the peak memory usage when operator i processes a unit data block, REG i Indicates the number of registers required by operator i to process a unit data block when it is running.

[0092] The total memory required by the operator group scheduled in parallel MEM tot The total number of registers required by the operator groups scheduled in parallel should not exceed the size of the SPM. tot The maximum number of registers in the PE data stream array should not be exceeded.

[0093] According to the operator topology, the data dependency between operators in the task is analyzed. The RISCV core in the host combines the operator runtime characteristics to divide the operators into three types of operator groups: operator groups with data dependency, operator groups that process the same data, and operator groups that process different data. Corresponding scheduling queues are maintained for the three types of operator groups: data dependency queue PQ1, data sharing queue PQ2, and data-independent queue PQ3, which are used to store the operator groups with data dependency, operator groups that process the same data, and operator groups that process different data obtained during task decomposition. The three data structures of data dependency queue PQ1, data sharing queue PQ2, and data-independent queue PQ3 are stored on DRAM.

[0094] Due to the data flow characteristics of the PE data flow array, the operator groups with data-dependent combinations and data-sharing have data flow relationships, which can effectively reduce the number of accesses to the SPM cache shared by PEs. The operator groups that are not related to data have no data flow and cannot utilize the data flow between PEs. At the same time, the operator groups that share data need to store the intermediate results of multiple operators, which increases the demand for memory capacity, while the operator groups that are data-dependent do not need it. Therefore, the priorities of the three queues are: data-dependent queue > data-sharing queue > data-independent queue. During the scheduling process, operators are scheduled in parallel from the queues according to this priority.

[0095] S22: In the operator topology graph, identify the dependency relationship between each operator and its successor operators.

[0096] Set the maximum number of parallel operators in the data-dependent operator group to MAXP. Take out the operators in the starting operator list list1 in sequence. Set the number of parallel operators in the current operator group Parr = 0, the current required number of registers regNum = 0, and the current required SPM size memSpm = 0.

[0097] like Figure 3As shown in the figure, if an operator has only one successor operator in the operator topology graph, the operator and the successor operator are combined into a data-dependent operator group. If the capacity and number of registers of the SPM are sufficient, the operator and the successor operator are merged. This can effectively organize and manage the dependencies between operators, reduce data transmission overhead, increase the parallelism and resource utilization of operator execution, and thus improve the overall computing performance.

[0098] Because only the final result is stored when executing a data-dependent operator group, other subsequent operators cannot obtain the result data generated by the current operator, resulting in repeated execution of the current operator. For the case of multiple subsequent operators, the result data generated by the current operator is stored once, and all subsequent operators are combined into a data-sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator. Figure 2 shown.

[0099] The specific operations are as follows.

[0100] If there is only one successor operator, calculate the number of registers currently required by the operator group:

[0101] regNum=regNum+REG i (i=0,1,…,i).

[0102] In the above formula, REG i Indicates the ith operator added to the operator group. The principle of this formula is to continuously search for nodes with in-degree 0 and out-degree 1, and add them to regNum to obtain the total number of registers required by the operator group.

[0103] If the average number of registers required by the operator (regNum+(Parr-1) / (Parr-1) is greater than the number of registers in the PE data stream array, the operator is added to the starting operator list list1; otherwise, the operator is added to the operator group, and the successor of the operator is continuously checked until the maximum number of registers in the PE data stream array is exceeded or the number of parallel operators Parr in the current operator group is equal to the maximum number of parallel operators MAXP or the operator has multiple successors. If the checked operator has multiple successors, the operator group is added to the data dependency queue PQ1, and step S23 is implemented for the current operator; otherwise, step S24 is implemented.

[0104] S23: If the operator has multiple successor operators in the operator topology graph, the result data generated by the current operator is stored at least once, and all the successor operators are combined into a data sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator. This reduces repeated calculation and transmission of data, optimizes the efficiency of system resource utilization, reduces the waste of memory and computing resources, and improves the overall execution speed and efficiency of the system.

[0105] Specifically, if the current operator has multiple successor operators, all subsequent operators are added to the ordered list of data sharing operators list2, and the first pointer ptr1 and the tail pointer ptr2 as well as the average PE utilization ULT of all operators in the ordered list of data sharing operators list2 are maintained. mean and the average PE utilization of the current operator group ULT cur The first pointer ptr1 points to the head of the ordered list list2 of the data sharing operator, and the tail pointer ptr2 points to the tail of the ordered list list2 of the data sharing operator.

[0106] If ULT cur >ULT mean , then add the operator pointed to by the first pointer ptr1 to the operator group and update ULT cur , regNum, memSpm and Parr. The first pointer ptr1 points to the next operator.

[0107] If ULT cur ≤ULT mean , then add the operator pointed to by the tail pointer ptr2 to the operator group and update the average PE utilization ULT of the current operator group cur , the number of registers currently required by the operator group regNum, the total memory occupied by the operator group memSpm and the number of parallel operators in the current operator group Parr; the tail pointer ptr2 points to the previous operator until the number of registers currently required by the operator group regNum exceeds the maximum number of registers in the PE data array, or the number of parallel operators in the current operator group Parr is equal to the maximum number of parallel operators MAXP, or the total memory occupied by the operator group memSpm is greater than the SPM capacity, or the position pointed to by the first pointer ptr1 is after the tail pointer ptr2. Clear the ordered list list2 of data sharing operators and implement step S24.

[0108] At runtime, the PE data stream array maintains the ordered list list2 of data sharing operators and the first pointer ptr1 and the last pointer ptr2 in the list. The ordered list list2 of data sharing operators is based on the PE utilization rate (UTL i ) Arrange the operators from largest to smallest.

[0109] When selecting operators in the data sharing operator group, based on the greedy method, the first pointer ptr1 and the tail pointer ptr2 of the data sharing operator ordered list list2 are dynamically updated to keep the average PE utilization of the operator group close to the average PE utilization of all operators in the data sharing operator ordered list2 (ULT mean ) to avoid low PE utilization of parallel operator groups. This optimizes the use efficiency of PE and ensures that the system can still run efficiently under high load.

[0110] S24: extract operators from the initial operator list list1, and implement step S22 or S23 according to the number of subsequent operators until the initial operator list list1 is empty.

[0111] S3: Schedule the parallel operator group according to the scheduling queue, and allocate computing resources to the parallel operators to be run in parallel in the parallel group based on the runtime characteristics. This step is as follows.

[0112] S31: Calculate the computing resources of each operator according to the proportion of the occupancy rate of the CAL functional components of each operator scheduled in parallel.

[0113] S32: Allocate a corresponding number of computing resources to operators in the operator group according to the calculated computing resources. In this way, computing resources can be reasonably allocated, and the number of processing units (PEs) can be dynamically adjusted according to the actual needs of the operators, thereby avoiding resource waste and improving the overall computing efficiency and resource utilization of the system.

[0114] Specifically, the parallel operator groups are scheduled according to the three obtained scheduling queues. For the PE data stream array, there are 16 PE computing resources thereon.

[0115] Since different operators need to perform different numbers of operations, the time required for operators to process data is different, so operators with fewer operations finish executing early and wait for operators with more operations, causing PE computing resources to run idle. At the same time, because PE utilization is related to the occupancy rate of CAL functional components during operator execution, in order to reduce the waste of computing resources and ensure the high efficiency of multi-operator parallelism, PE computing resources will be allocated according to the proportion of the occupancy rate of CAL functional components of each operator scheduled in parallel.

[0116] The computing resource allocation formula for parallel scheduling of each operator is as follows:

[0117] OCC caltot =OCC1+OCC2+…+OCC Parr .

[0118] Resouce i =max(OCC i / OCCcaltot *16,1).

[0119] Among them, Parr represents the number of parallel operators in the operator group, OCC caltot Indicates the total CAL functional component occupancy rate of the operator group, OCC i represents the occupancy rate of the CAL functional components of operator i, Resource i It indicates the number of PE computing resources allocated to operator i, and 16 is the maximum number of PEs in the PE data flow array.

[0120] According to the calculated computing resources of each operator, the corresponding number of PEs are allocated to the operators in the operator group in turn. Since the data flow distance between PEs has the characteristics of the optimal substructure when the data flow graph is constructed, in order to make the data flow distance between PEs as close as possible, a greedy strategy is used to allocate PE computing resources to operators. In order to facilitate programming and subsequent dynamic reconstruction of the data flow graph, this method will allocate computing resources to each parallel operator in turn according to the PE number.

[0121] The present invention calculates the occupancy rate OCC of each operator CAL functional component of parallel scheduling i Occupancy rate of the total CAL function components of the operator group OCC caltot The ratio of is used to calculate the number of PE computing resources allocated to each operator i. i To ensure that each operator can be allocated PE computing resources. i The minimum value of is set to 1, which means at least one PE is used to execute the operator. 16 is the total number of PE computing resources in the PE data stream array.

[0122] S4: The steps of dynamically reconstructing the original operator data flow graph of the parallel operator group according to the operator data flow graph are as follows.

[0123] When multiple operators are executed in parallel, the corresponding subtasks of each operator need to be spliced, which will involve changes in the data flow between PEs. Therefore, for PE computing resources that execute the same operator, the data flow between PEs does not change.

[0124] According to the PE computing resources allocated to each operator in step S3, the original data flow graph of each operator is mapped to the new data flow graph in turn. For the PE computing resources executing different operators, dynamic reconstruction is performed according to the type of the scheduled operator group.

[0125] When scheduling data-independent operator groups, no data flow relationship is established between PE computing resources. When scheduling data-identical operator groups or data-dependent operator groups, a data flow relationship is established between PE computing resources, such as Figure 4At the same time, compared with the computing resources of the entire PE data flow array occupied by each operator before the data flow graph is spliced, the PE computing resources occupied by each operator after the data flow graph is spliced ​​will be scaled. Therefore, it is necessary to dynamically adjust the number of instructions to be executed by the operator on each PE computing resource according to the number of PE computing resources to produce correct results.

[0126] like Figure 4 As shown, three operator topology diagrams of operators 1 to 5 are shown.

[0127] In the first operator topology, a data dependency group is constructed: the result data generated by operator 1 is directly passed to the subsequent operator 2 without being stored.

[0128] In the second operator topology, in order to execute operator 3, operator 1 needs to be re-executed, and operator 3 is executed repeatedly.

[0129] In the third operator topology, first extract the result data of operator 1. Construct a data sharing group: Operator 1 does not need to be executed repeatedly. The number of accesses to the three operators: 3→1.

[0130] S5: After the PE data stream array completes the parallel execution of the operator, check whether the three queues are all empty. If so, terminate the operator scheduling. Otherwise, return to step S3.

[0131] Specifically, if the currently scheduled operator group is a data-dependent operator group, and the total data generated by the operator group is less than the SPM capacity, the subsequent data-dependent operator group is obtained through step S3, inserted into the head of the queue of the data-dependent operator group, and returns to step S3. Otherwise, directly return to step S3 until all queues are empty.

[0132] The data generated by the operator group can be directly stored in the SPM without having to be stored in DRAM. If there are subsequent operators, the operators can be directly scheduled to use the data in the SPM for calculation without having to load the data from DRAM.

[0133] If the three operator group scheduling queues dynamically maintained by the runtime system are all empty, it means that all nodes in the operator topology graph have been executed, the task execution is completed, and the operation of the PE data flow array is ended.

[0134] like Figure 5 As shown, when the data flow accelerator executes this step, in the case of sequential execution of a single operator, the operator subtask00 is divided into subtask01 and subtask02. In this figure, the operator subtask01 is divided into 4 subtasks: subtask10, subtask11, subtask12 and subtask13.

[0135] In the case of multiple operators executing in parallel, operator subtask00 is divided into subtask01 and subtask02; operator 2 of subtask10 is divided into three subtasks. In the original solution, the data flow accelerator will schedule operators one by one and execute the subtasks divided from them in sequence. The instruction templates of operators subtask00 and subtask10 are merged, subtask11 is merged, and different operator groups are deployed on the data flow accelerator. Among them, the instruction template of newsubtask0 is deployed on the PE data flow array in the data flow accelerator.

[0136] In the proposed multi-operator parallel execution method, the present invention provides a subtask dynamic splicing method, which splices the same subtask of different operators into a new subtask, thereby forming a new data flow graph, and deploys it on a data flow accelerator, so that different operators can realize multi-operator parallelism at the subtask granularity.

[0137] Example 2

[0138] This embodiment provides a specific application case to achieve fine-grained parallel scheduling of multiple operators combining software and hardware. In market applications, the system can be used to perform efficient task scheduling on the DFU accelerator by dividing operators into three different types of operator groups for scheduling, making full use of the characteristics of DFU accelerator data flow, reducing the access overhead to the SPM on the DFU, and improving the utilization of PE computing resources.

[0139] The implementation steps of the present invention are described by taking the Unet network as an example.

[0140] First, the Unet network is analyzed to obtain the operator topology diagram between multiple operators during task execution.

[0141] Figure 6 The figure shows the operator topology in a deep learning model; in particular, it shows the entire process of data flow and processing in a deep learning model. A portion of the Unet network starts with the activation function ReLU, which introduces nonlinearity to the network by setting all negative values ​​to zero, allowing the Unet network to capture more complex features and patterns. Here, only a portion of the Unet network is scheduled for operators because: (1) the overall computational graph of the Unet network is very large and cannot be displayed in a picture within a limited space, or after being displayed, the nodes in the computational graph will be very small and difficult to see; (2) the purpose here is to perform multi-operator scheduling on a portion of the Unet network to ensure that the scheduling method is feasible and correct.

[0142] Next, the data passes through four maximum pooling layers (MaxPool). This continuous pooling operation significantly reduces the spatial size of the feature map while retaining key information, which helps to improve the computational efficiency of the model and reduce the risk of overfitting.

[0143] The data then flows into four consecutive convolutional layers (Conv), which extract features of the input data through convolution kernels and map these features to a new feature space. After the convolutional layer, the Unet network includes two shape operations (Shape), which may be used to adjust the dimensions of the feature maps to ensure that they match the input requirements of subsequent layers. Next is the slicing operation (Slice), which may split the feature map into multiple parts for independent processing or extraction of specific features.

[0144] These segmented features are then merged through a concatenation operation (Concat), which helps integrate features from different layers or different parts and enhances the feature fusion capability of the Unet network. After that, the model includes three consecutive resize operations (Resize), which may be used to adjust the size of the feature map to a specific output size or to adapt to the input requirements of subsequent layers. Finally, all adjusted feature maps are merged through the final concatenation operation (Concat) to provide a comprehensive feature representation for the final decision or classification of the Unet network.

[0145] Overall, the operator topology diagram shows a series of processing steps from input to output of data in a deep learning model, covering multiple key operations such as activation, pooling, convolution, reshape adjustment, slicing, splicing and resizing. These operations work together to extract and fuse features, providing a basis for completing complex classification and recognition tasks.

[0146] Figure 7 The data flow diagram of the activation function Relu and the maximum pooling layer MaxPool of the Unet network is shown. In the activation function layer Relu, the PEs in the first row are assigned to calculate the activation function Relu, and the rest of the PEs are idle. In the maximum pooling layer MaxPool, the PEs in the first and second rows are assigned to perform pooling, and the PEs in the third and fourth rows are idle.

[0147] like Figure 8 As shown, the operator groups are classified according to the data flow graph on the configured PE data flow array. For example, in the Unet network, the activation function Relu and the maximum pooling layer MaxPool can be constructed into an operator group that shares data. In other parts of the Unet network calculation, data-dependent operator groups and data-independent operator groups can be constructed, and the corresponding priority queues can be maintained.

[0148] Dynamically allocate computing resources of the PE data flow array to different operator groups according to the operator group priority queue. In order to make full use of PE computing resources, PE computing resource allocation calculations are performed. Dynamically reconstruct the multi-operator parallel data flow graph at the subtask level to perform data flow scheduling and multi-operator parallelism. This case takes the data shared operator group composed of the data output by the operator of the activation function Relu and the operators of the four maximum pooling layers MaxPool as an example. The operator group is taken out from the corresponding priority queue, and the memory address of the data output by the activation function Relu and the data flow graph of the operators of the four maximum pooling layers MaxPool are obtained through the stored data structure. The data flow graph of the activation function Relu and the maximum pooling layer MaxPool is reconstructed, and then the reconstructed operators of the four maximum pooling layers MaxPool are scheduled. Fig. 9 As shown. Fig. 9 In the example, SPM extracts the result data of the activation function Relu. These data are then sent to the operators of the four maximum pooling layers MaxPool formed after reconstruction. The operators of the first maximum pooling layer MaxPool1 formed after reconstruction are responsible for performing the first maximum pooling operation; the operators of the second maximum pooling layer MaxPool2 further reduce the spatial size of the feature map based on the first pooling; the operators of the third maximum pooling layer MaxPool3 continue to perform pooling operations to extract more abstract features; the operators of the fourth maximum pooling layer MaxPool4 provide highly abstract feature representations for subsequent layers of the network. By reconstructing and scheduling these operators, it can be ensured that each PE is fully allocated and utilized, thereby improving the computing efficiency of the entire system. The data flow graph reconstruction and scheduling of other operator groups are similar to this process.

[0149] It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention specification and its drawings are illustrative and do not constitute a limitation on the claims. The scope of protection of the present invention is defined by the claims and their equivalents. The present invention specification contains multiple inventive concepts, such as "preferably" and "according to a preferred embodiment", which means that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application based on each inventive concept.

Claims

1. A fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture, characterized in that: The method comprises: Establish an operator topology graph based on the decomposed operators related to the task scheduling process, and construct an operator data flow graph; Analyzing the data dependency relationship between operators in the task according to the operator topology graph, grouping the operators based on the data dependency relationship and the runtime characteristics of the operators, and maintaining a corresponding scheduling queue for each operator group; Scheduling the parallel operator group according to the scheduling queue, and allocating computing resources to the parallel operators to be run in parallel in the parallel group based on the runtime characteristics; Dynamically reconstructing the original operator data flow graph of the parallel operator group according to the operator data flow graph; After the PE data stream array completes the parallel execution of the operator, it is checked whether all the scheduling queues are empty. If so, the operator scheduling is terminated. Otherwise, the parallel operator group is continuously scheduled according to the scheduling queue.

2. The fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture according to claim 1 is characterized in that: The method of constructing an operator data flow graph on a PE data flow array includes: Divide the decomposed operators into code segments; Generate subtasks corresponding to each code segment; The PE data stream array executes subtasks in sequence.

3. The fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture according to claim 1 or 2 is characterized in that: The method for grouping operators based on the data dependency and the runtime characteristics of the operators includes: Divide operators into groups with data dependencies, groups of operators that process the same data, and groups of operators that process different data; In the operator topology graph, the dependency relationship between each operator and its successor operator is identified. If an operator has only one successor operator in the operator topology graph, the operator and the successor operator are combined into a data-dependent operator group. If the capacity and number of registers of the SPM are sufficient, the operator and the successor operator are merged.

4. The fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture according to any one of claims 1 to 3, characterized in that: The method for grouping operators based on the data dependency and the runtime characteristics of the operators also includes: If an operator has multiple successor operators in the operator topology graph, the result data generated by the current operator is stored at least once, and all successor operators are combined into a data sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator.

5. The fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture according to any one of claims 1 to 4, characterized in that: The method for grouping operators based on the data dependency and the runtime characteristics of the operators also includes: When selecting operators in the data sharing operator group, based on the greedy method, the head pointer and tail pointer of the ordered list of data sharing operators are dynamically updated to keep the average PE utilization of the operator group close to the average PE utilization of all operators in the ordered list of data sharing operators, so as to avoid the PE utilization of the parallel operator group being too low.

6. The fine-grained multi-operator parallel scheduling method based on heterogeneous data flow architecture according to any one of claims 1 to 5, characterized in that: The method for allocating computing resources to parallel operators includes: The computing resources of each operator are calculated according to the proportion of the occupancy rate of the CAL functional components of each operator scheduled in parallel; A corresponding number of processing units are allocated to the operators in the operator group according to the calculated computing resources.

7. A fine-grained multi-operator parallel scheduling system based on a heterogeneous data flow architecture, comprising a processor, characterized in that: The processor is configured to: Establish an operator topology graph based on the decomposed operators related to the task scheduling process, and construct an operator data flow graph; Analyzing the data dependency relationship between operators in the task according to the operator topology graph, grouping the operators based on the data dependency relationship and the runtime characteristics of the operators, and maintaining a corresponding scheduling queue for each operator group; Scheduling the parallel operator group according to the scheduling queue, and allocating computing resources to the parallel operators to be run in parallel in the parallel group based on the runtime characteristics; Dynamically reconstructing the original operator data flow graph of the parallel operator group according to the operator data flow graph; After the parallel execution of the operator is completed, it is checked whether each of the scheduling queues is empty. If so, the operator scheduling is terminated, otherwise, the parallel operator group is continuously scheduled according to the scheduling queue.

8. The fine-grained multi-operator parallel scheduling system based on heterogeneous data flow architecture according to claim 7 is characterized in that: The processor is further configured to: When performing the step of grouping operators based on the data dependency and the runtime characteristics of the operators, the operators are divided into operator groups with data dependencies, operator groups processing the same data, and operator groups processing different data; In the operator topology graph, the dependency relationship between each operator and its successor operator is identified. If an operator has only one successor operator in the operator topology graph, the operator and the successor operator are combined into a data-dependent operator group. If the capacity and number of registers of the SPM are sufficient, the operator and the successor operator are merged.

9. The fine-grained multi-operator parallel scheduling system based on heterogeneous data flow architecture according to claim 7 or 8, characterized in that: The processor is further configured to: When executing the step of grouping operators based on the data dependencies and the runtime characteristics of the operators, if the operator has multiple successor operators in the operator topology graph, the result data generated by the current operator is stored at least once, and all the successor operators are combined into a data sharing operator group to avoid repeated execution of the current operator and repeated loading of the result data generated by the current operator.

10. The fine-grained multi-operator parallel scheduling system based on heterogeneous data flow architecture according to any one of claims 7 to 9, characterized in that: The processor is further configured to: In the step of grouping operators based on the data dependencies and the runtime characteristics of the operators, when selecting operators in the data sharing operator group, based on a greedy method, the head pointer and the tail pointer of the ordered list of data sharing operators are dynamically updated to keep the average PE utilization of the operator group close to the average PE utilization of all operators in the ordered list of data sharing operators, so as to avoid too low PE utilization of the parallel operator group.

Images