Program optimization method and device and related product

Abstract

The invention provides a program optimization method and device and a related product, a compiler can determine at least one pipeline level in a software pipeline according to a data dependency relationship of nodes and resources used by the nodes so as to determine the software pipeline for program operation, so that when a computing device operates a program, the software pipeline is not required to be subjected to the program operation, and the operation efficiency is improved. Different resources of the computing device can run different pipeline levels in the program in parallel according to the software pipeline, so that the resources of the computing device are fully utilized, and the processing efficiency of the computing device is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a program optimization method, device and related products. Background Art

[0002] In the computer field, instruction pipeline is a technology used to implement instruction-level parallelism within a hardware processing device. Pipeline divides the incoming instructions into a series of pipeline stages so that different units of the hardware device (such as a processor) execute these pipeline stages in parallel, thereby keeping each part of the hardware device busy. Instruction pipeline can include software pipeline and hardware pipeline.

[0003] Among them, software pipelining can optimize the performance of the program. Software pipelining is a form of out-of-order execution, which reorders program instructions through a compiler so that different units of a hardware device work in parallel, rather than implementing instruction parallelism through hardware devices.

[0004] However, due to data dependencies and resource constraints, how to implement software pipelines to maximize the use of available resources of hardware devices at the same time has become a problem worthy of attention. Summary of the invention

[0005] Based on the existing technology, the present disclosure provides a program optimization method, device and related products, which can realize software pipeline and fully utilize the resources of hardware processing devices, thereby improving the efficiency of calculation.

[0006] The present disclosure provides a program optimization method, the method comprising:

[0007] Obtaining a directed acyclic graph corresponding to the program, wherein the directed acyclic graph includes a plurality of nodes;

[0008] Dividing the directed acyclic graph into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes;

[0009] Each subgraph serves as a pipeline stage, and the pipeline stage is used to determine the pipeline of the program operation, so that different resources of the computing device can run different pipeline stages in the program in parallel according to the pipeline.

[0010] The present disclosure also provides a program optimization device, which includes a memory and a processor. The memory stores a computer executable program. When the computer executable program is executed by the processor, the program optimization method as described above is implemented.

[0011] The present disclosure also provides a computer program product, including a computer program or instructions, and when the computer program or instructions are executed by a processor, the above-mentioned program optimization method is implemented.

[0012] The present disclosure also provides a computer-readable storage medium, wherein the computer storage medium is used to store a computer program or instruction, and when the computer program or instruction is executed by a processor, the above-mentioned program optimization method is implemented.

[0013] The program optimization method, device and related products provided by the present disclosure can divide the directed acyclic graph into multiple subgraphs according to the data dependency of the nodes and the resources used by the nodes, wherein each subgraph represents a pipeline stage in the software pipeline, wherein the multiple pipeline stages obtained by dividing the directed acyclic graph can be executed by the computing device in the order of their data dependency. The multiple pipeline stages can be used to determine the software pipeline of the program operation, so that when the computing device runs the program, different resources of the computing device can run different pipeline stages in the program in parallel according to the above-mentioned software pipeline, thereby making full use of the resources of the computing device and improving the processing efficiency of the computing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic diagram of an embodiment of a software pipeline;

[0015] Figure 2 This is a schematic diagram of an application scenario of an embodiment of the present disclosure;

[0016] Figure 3 for Figure 2 A hardware architecture diagram of an embodiment of a computing device;

[0017] Figure 4 A flowchart of a program optimization method according to an embodiment of the present disclosure;

[0018] Figure 5 A schematic diagram of a software pipeline generated by a program optimization method according to an embodiment of the present disclosure;

[0019] Figure 6 A flowchart of a pipeline level determination method in a program optimization method according to an embodiment of the present disclosure;

[0020] Figure 7 A schematic diagram of a directed acyclic graph and a method for determining a pipeline level thereof according to an embodiment of the present disclosure;

[0021] Figure 8 A schematic diagram of a directed acyclic graph and a method for determining a pipeline level thereof according to an embodiment of the present disclosure;

[0022] Fig. 9 for Figure 8 The schematic diagram of the software pipeline determined by the directed acyclic graph shown;

[0023] Fig.10 The present invention is a flowchart of a program optimization method according to another embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0025] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms "a", "said" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0026] It should be understood that the term "and / or" used in this article is only a description of the same field of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0027] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present disclosure, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0028] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0029] In the computer field, instruction pipeline is a technology used to implement instruction-level parallelism within a hardware processing device. Pipeline divides the incoming instructions into a series of pipeline stages so that different units of the hardware device (such as a processor) execute these pipeline stages in parallel, thereby keeping each part of the hardware device busy. Instruction pipelines can include software pipelines and hardware pipelines. Among them, software pipelines are a form of out-of-order execution, which reorders program instructions through a compiler to optimize the performance of the program so that different units of the hardware device work in parallel, rather than implementing instruction parallelism through hardware devices.

[0030] In an implementation example of a software pipeline, the resources of the hardware device may include at least one computing unit (i.e., computing resources), at least one level of memory (i.e., storage resources), and at least one access unit (i.e., memory access resources). When executing a task, the hardware device needs to sequentially perform data loading operations, data computing operations, and data saving operations, and the above different operations may use different resources of the hardware device. The program executed on the hardware device may be a loop program containing the following operations, which need to be executed 3 times in a loop:

[0031] Operation 1Load a;

[0032] Operation 2Load b;

[0033] Operation 3c = a + b;

[0034] Operation 4store c.

[0035] The compiler can divide each operation in the above program into different pipeline stages, and determine the software pipeline of the program according to the pipeline stages obtained after the division. The structure of the software pipeline can be as follows Figure 1 As shown. Figure 1 In the software pipeline shown, the X direction represents the operations that need to be performed in order for one iteration, which may include multiple pipeline stages. Figure 1 Three pipeline stages including load operations, calculation operations, and save operations are shown. The Y direction represents operations that can be performed in parallel by the hardware device, such as the calculation operation in the first iteration and the load operation in the second iteration use different resources of the hardware device, and the hardware device can perform calculation operations (such as the above operation 3) and load operations (such as the above operations 1 and 2) in parallel. The save operation in the first iteration, the calculation operation in the second iteration, and the load operation in the third iteration use different resources of the hardware device, respectively, and the hardware device can perform the above save operation (such as the above operation 4), calculation operation (such as the above operation 3), and load operation (such as the above operation 1 and operation 2) in parallel.

[0036] With the development of big data and machine learning, the demand for the computing power of hardware processing devices has gradually increased, so computer systems formed by multi-core processors or multiple hardware devices have become a trend. In a computer system with multi-core processors or multiple hardware devices working together, how to schedule the computing operations in the program in the computer system with multi-core processors or multiple hardware devices working together to improve the parallelism of program operation has become an important issue. At this time, the design of the software pipeline not only needs to consider the utilization of the storage resources and computing resources of the hardware device (such as the software pipeline of the above-mentioned implementation example), but also needs to consider the parallel utilization rate of the computing resources of the hardware device. At this time, how to implement the software pipeline design of the program is the technical problem to be solved by the technical solution disclosed in this disclosure.

[0037] In order to illustrate the technical solution of the present disclosure, the present disclosure exemplarily illustrates its application scenario. Figure 2 As shown, the above-mentioned hardware device may be a heterogeneous hardware device 200 including a computing device 210 and a processing device 230 , and the computing device 210 and the processing device 230 have different instruction set architectures.

[0038] The computing device 210 is configured to perform operations specified by the user to perform deep learning or machine learning calculations, and can interact with the processing device 230 through the interface device 220 to jointly complete the tasks specified by the user (including but not limited to image recognition, speech recognition, etc.). The above tasks may include a variety of calculations, including but not limited to scalar operations, vector operations, and matrix operations, etc., to achieve the corresponding tasks.

[0039] The processing device 230, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 210. Depending on the implementation, the processing device 230 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or computing devices, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 210 of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 210 and the processing device 230 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0040] The interface device 220 is used to transmit data and control instructions between the computing device and the processing device. The storage device 240 is used to store data to be processed, which can be a DDR memory (DRAM) for storing data of the computing device 210 and / or the processing device 230. The storage device 240 can be recorded as an external storage resource of the computing device 210.

[0041] Furthermore, the computing device 210 may include a scalar computing unit for performing scalar operations and a tensor computing unit for implementing tensor operations of different dimensions such as vector operations and matrix operations. The scalar computing unit and the tensor computing unit of the computing device may be recorded as computing resources of the computing device. The tensor computing unit of the computing device may be a multi-core processor including multiple processor cores, and each processor core may be recorded as a computing resource. Each processor core is used to implement a variety of different computing operations such as vector operations and matrix operations. For example, each processor core may include a vector operation module and a matrix operation module to implement corresponding operations.

[0042] Furthermore, each processor core may also include a storage module for storing data required for calculations, and the storage module may be recorded as a first-level storage resource of the computing device. Each processor core may also include a memory access module, which is used to interact with external storage resources or other processor cores on the computing device. For example, each processor core may include a move direct memory access module (MVDMA) and an input / output direct memory access module (IODMA), MVDMA is used for the processor core to interact with other processor cores on the computing device, and IODMA is used for the processor core to interact with external storage resources of the computing device. Among them, the memory access module in each memory core can be recorded as a memory access resource of the computing device.

[0043] Furthermore, multiple processor cores of a computing device can form multiple clusters, and multiple clusters can communicate data with each other, and each cluster can also be recorded as a computing resource. By setting up multiple processor cores, the parallelism of the program can be improved, thereby improving the processing efficiency of the processor. Figure 3 As shown, four clusters 310 are shown in the figure, where the four clusters are marked as cluster 1 to cluster 4. With the development of hardware, the cluster of the computing device of the present disclosure may also include 8, 16, 64 or even more clusters. Each cluster 310 includes multiple processor cores 320 and a storage core 330. The present disclosure does not limit the number of processor cores 320. Figure 3 As shown, four processor cores 320 are shown in the figure as an example, and the four processor cores are respectively labeled as processor core 1 to processor core 4.

[0044] Continue to see Figure 3 The storage core 330 can be used to implement data exchange between multiple processor cores in the same cluster, and can also be used between clusters or between clusters and external storage devices (such as Figure 2 The storage core 330 can be recorded as a second-level storage resource of the computing device.

[0045] The storage core 330 includes SRAM 331, broadcast bus, cluster direct memory access module (cluster direct memory access, CDMA) 333 and global direct memory access module (global direct memory access, GDMA) 332. SRAM 331 plays the role of a high-performance data transfer station. The data reused between different processor cores in the same cluster does not need to be obtained from DRAM by each processor core, but is transferred between processor cores through SRAM. The storage core only needs to quickly distribute the reused data from SRAM to multiple processor cores to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access. CDMA 333 is used to control the memory access of SRAM 331 between different clusters 310 in the same computing device. GDMA 332 cooperates with the external storage controller to control the memory access from SRAM 331 of cluster 310 to off-chip memory 240, or read data from off-chip memory 240 to SRAM 331. Optionally, the CDMA 333 and the GDMA 332 in the storage core 330 may be implemented in the same component, and the CDMA 333 and the GDMA 332 in the storage core 330 may be recorded as memory access resources of the computing device.

[0046] Among them, the data interaction within the computing device can be recorded as a MOVE operation. For example, the MOVE operation may include: data interaction between each processor core or data interaction between each cluster. Generally, the resources used by the MOVE operation are memory access resources on the computing device, and the memory access resources can be CDMA333 or MVDMA inside the processor core. The data interaction between the computing device and the external device can be recorded as a load operation or a save operation. The load operation or the save operation also needs to use the memory access resources of the computing device, and the memory access resources can be the above-mentioned GDMA332 or IODMA inside the processor core. The resources used by each operation can be determined according to the specific operation.

[0047] Optionally, a compiler can be run on the processing device. Among them, the compiler can be implemented as a computer program for converting a source program written by a user into an executable file (such as a binary file) that can be executed by a computing device. Further, in the embodiment of the present disclosure, the compiler can expand the program with a loop structure running on the computing device during the compilation stage, and optimize the program to obtain a software pipeline for running the program on the computing device, so as to generate an executable file that can be executed by the computing device according to the software pipeline. Then, the executable file compiled and generated by the compiler can be transmitted to the computing device through the interface device 220, so that the computing device can execute the above-mentioned executable file to achieve the corresponding computing task. Among them, the executable file includes but is not limited to computing instructions and data handling instructions. In this way, when the computing device runs the program, different computing units of the computing device can execute the operations in the program in parallel, thereby improving the computing efficiency of the program.

[0048] like Figure 4 As shown, an embodiment of the present disclosure provides a program optimization method, which can be applied to a compiler to divide the operations in the program into at least one pipeline stage to facilitate the implementation of a software pipeline, so that an executable file that can be directly executed by a computing device can be generated according to the software pipeline. The method of the embodiment of the present disclosure may include:

[0049] S410, obtaining a directed acyclic graph corresponding to the program, wherein the directed acyclic graph includes a plurality of nodes;

[0050] The program of the embodiment of the present disclosure may refer to a computer code including a series of computing operations, which can be run on a computing device to implement certain functions, such as image recognition, speech recognition, etc., which are not specifically limited here. Optionally, the above program may adopt a loop structure, and the above program may include iteration n times, where n is greater than or equal to 1 and n is an integer.

[0051] In the disclosed embodiment, the compiler can obtain the above program, analyze the above program, determine the data dependency between the operations in the program, and obtain the directed acyclic graph corresponding to the program according to the data dependency of the operations in the program. The operations in the program may include calculation operations and memory access operations, and the calculation operations may be used to complete scalar operations, vector operations, and matrix operations, etc. The memory access operations are used to complete data transfer, including but not limited to load operations, save operations, and MOVE operations.

[0052] Optionally, the directed acyclic graph can be a directed acyclic graph corresponding to one iteration in the program. The directed acyclic graph consists of some nodes and ordered edges, and each edge has a direction. Each node can be used to indicate an operation. Each edge in the directed acyclic graph can be used to represent the data dependency between nodes. In a directed acyclic graph, each edge points from one node to another node, and each node has an in-degree and an out-degree; the number of edges pointing to the node is recorded as the in-degree of the node; the number of edges emitted from the node is called the out-degree of the node.

[0053] S420. Divide the directed acyclic graph into at least one subgraph according to the data dependencies of the nodes and the resources used by the nodes; wherein each subgraph may include at least one node, and each subgraph may serve as a pipeline level, so that the compiler can determine the software pipeline of the program according to the above pipeline levels.

[0054] The resources used by a node may refer to the computing units occupied by a computing device to run the node, which may be at least one processor core or scalar computing unit on the computing device. The resources used by a node may also refer to the memory access resources occupied by the node to run, such as CDMA, GDMA, IODMA or MVDMA of the computing device.

[0055] Optionally, the resources used by the nodes can be recorded in the directed acyclic graph using specific identifiers, and the specific identifiers can be recorded corresponding to the corresponding nodes. For example, different colors are used to record the resources used by each node in the directed acyclic graph; or, strings are used to record the resources used by each node in the directed acyclic graph. For another example, a resource reservation table is used to record the resources used by each node in the directed acyclic graph. This is only an example description and is not specifically limited.

[0056] In the disclosed embodiment, the compiler can divide the directed acyclic graph into multiple subgraphs according to the data dependency of the nodes and the resources used by the nodes, wherein each subgraph represents a pipeline stage in the software pipeline, wherein the multiple pipeline stages obtained according to the division of the directed acyclic graph can be executed by the computing device in the order of their data dependency. The multiple pipeline stages can be used to determine the software pipeline of the program operation, so that when the computing device runs the program, different resources of the computing device can run different pipeline stages in the program in parallel according to the above-mentioned software pipeline, thereby making full use of the resources of the computing device and improving the processing efficiency of the computing device.

[0057] Furthermore, the above method may also include a pipeline generation step, which may specifically include:

[0058] S430: Automatically determine a pipeline for program execution according to at least one pipeline stage corresponding to the directed acyclic graph.

[0059] The program may adopt a loop structure, and the program may include iterations n times, where n is greater than or equal to 1 and is a positive integer. The compiler may expand multiple iterations of the program to improve the utilization of the hardware computing device. Specifically, the compiler may automatically determine the pipeline of the program according to the number of iterations of the program and at least one pipeline stage included in the above iteration, so that when the computing device runs the program, different resources of the computing device can execute different operations in the program in parallel, thereby improving the processing efficiency of the computing device.

[0060] like Figure 5 As shown, Figure 5 The software pipeline shown is determined by the compiler according to the operations in the program. The X direction represents the operations that need to be executed in sequence for one iteration, which may include m (m is a positive integer greater than or equal to 1) pipeline stages, and the Y direction represents the operations that can be executed in parallel by the computing device. In the embodiment of the present disclosure, the compiler can divide one iteration of the program into m pipeline stages according to the directed acyclic graph corresponding to the program, and the m pipeline stages are executed in order according to their data dependencies, and each pipeline stage can contain at least one node. Afterwards, the compiler can determine the following based on the m pipeline stages and the number of iterations of the program: Figure 5 The software pipeline shown in the figure is used to generate an executable file that can be executed by the computing device according to the software pipeline. In this way, when the computing device executes the program, different resources of the computing device can execute different pipeline stages in parallel at the same time. For example, the computing device can execute Stage 2 of the first iteration and Stage 1 of the second iteration in parallel, thereby improving the parallelism and computing efficiency of program execution.

[0061] In one embodiment, in the step of determining the pipeline level by dividing the directed acyclic graph, the compiler may traverse the directed acyclic graph. For example, the compiler may traverse the directed acyclic graph in a depth-first manner to determine the pipeline level corresponding to the program. Figure 6 As shown, the above step of determining the pipeline level by dividing the directed acyclic graph may include:

[0062] S610, determining whether the current node and the predecessor node of the current node use the same resources according to the data dependency relationship of the nodes;

[0063] If the current node and the predecessor node of the current node use the same resources, step S620 is executed to divide the current node into the subgraph to which the predecessor node of the current node belongs.

[0064] If the current node and the predecessor node of the current node use different resources, step S630 is executed to divide the current node and the predecessor node of the current node into different subgraphs.

[0065] The current node can refer to any node in the directed acyclic graph. The predecessor node of the current node refers to a node that has a data dependency relationship with the current node. The output data of the predecessor node of the current node can be used as the input data of the current node. The current node needs to start executing after the predecessor node of the current node is executed. If the current node is the starting node of the directed acyclic graph, the current node has no predecessor node.

[0066] The compiler may traverse the entire directed acyclic graph starting from the current node until determining the subgraph to which each node belongs. Specifically, the compiler may determine the predecessor node of the current node based on the data dependency relationship between the nodes, wherein the compiler may first determine the subgraph to which the predecessor node of the current node belongs, and then determine the subgraph to which the current node belongs.

[0067] If the current node has a predecessor node, and the current node and its predecessor node need to use the same resources, it means that the current node and its predecessor node should be executed in the same resource order of the computing device, and the compiler can divide the current node and its predecessor node into the same subgraph (the same pipeline level). Specifically, the compiler can divide the current node into the subgraph to which its predecessor node belongs. Among them, multiple nodes in the same pipeline level are executed in the same resource order of the computing device according to their data dependency.

[0068] If the current node has a predecessor node, and the current node and its predecessor node need to use different resources, it means that in different iterations of the program, the current node and its predecessor node can be executed in parallel by different resources of the computing device, and the compiler can divide the current node and its predecessor node into different subgraphs (different pipeline levels). In different iterations of the program, different pipeline levels can be executed in parallel by different resources of the computing device.

[0069] In the conventional pipeline level division method, the compiler can use each node in the directed acyclic graph as a pipeline level, but too many pipeline levels will cause the startup and emptying process to be too long, affecting the running time of the program, and too many pipeline levels will increase the amount of resources requested, which may cause the inability to generate code. Compared with the conventional pipeline level division method, the embodiment of the present disclosure can reduce the number of pipeline levels by determining whether nodes with data dependencies use the same resources to implement pipeline level operations, so as to achieve the optimal execution time of the program, thereby improving the processing efficiency of the computing device for the program.

[0070] Optionally, the current node is a node with an in-degree of 0 in the directed acyclic graph, that is, the compiler traverses the directed acyclic graph from the start node of the directed acyclic graph until the subgraph to which each node belongs is determined. The specific traversal method can be referred to above. This ensures that the data dependency between nodes is correct and the number of pipeline stages can be reduced as much as possible.

[0071] Optionally, there are multiple current nodes; the compiler can determine the pipeline stage to which each current node belongs. If there is no data dependency between the multiple current nodes, and each current node uses different resources from its predecessor node, the compiler can divide each current node into a subgraph (pipeline stage), that is, each current node can be used as a separate pipeline stage.

[0072] Furthermore, in order to reduce the number of pipeline stages, the above pipeline stage division operation may also include:

[0073] If there is no data dependency between multiple current nodes, the compiler can divide the multiple current nodes into the same subgraph.

[0074] Specifically, the compiler traverses the entire directed acyclic graph from the node with in-degree 0 (i.e., the current node). If there is no data dependency between multiple current nodes, the multiple current nodes are divided into the same subgraph. For example, if multiple current nodes use different resources and there is no data dependency between multiple current nodes, the compiler can divide multiple current nodes into the same subgraph (i.e., one pipeline stage).

[0075] Furthermore, after determining the subgraph to which the current node belongs, the compiler can update the in-degree of the successor node of the current node. The successor node of the current node refers to a node that has a data dependency with the current node and starts executing after the current node is executed. The output data of the current node is the input data of its successor node. Specifically, after determining the subgraph to which the current node belongs, the compiler can subtract 1 from the in-degree of the successor node of the current node. Afterwards, the compiler can continue to traverse the current node with an in-degree of 0 in the directed acyclic graph, and repeat the above steps until the subgraph partitioning of all nodes is completed.

[0076] The pipeline stage division operation of the embodiment of the present disclosure is described below with reference to the accompanying drawings. Figure 7 The directed acyclic graph corresponding to the program is shown. Figure 7 As shown, the directed acyclic graph includes node 1, node 2, node 3, node 4 and node 5, wherein node 1 and node 2 use the same resources, for example, node 1 and node 2 need to use the memory access resources of the computing device to implement the loading operation, and node 3, node 4 and node 5 use different resources respectively, for example, node 3 uses the cluster 2 of the computing device to perform a computing operation, node 4 uses the cluster 3 of the computing device to perform another computing operation, and node 5 uses the memory access resources of the computing device to implement the saving operation.

[0077] At the initial moment, the in-degree of node 1 is 0 and the out-degree is 2; the in-degree of nodes 2, 3 and 4 is 1 and the out-degree is 1; the in-degree of node 5 is 2 and the out-degree is 0. The embodiment of the present disclosure can use the node with in-degree 0 as the current node.

[0078] Among them, node 1 is the current node, and the compiler can traverse the directed acyclic graph starting from node 1 and divide node 1 into a subgraph, for example, node 1 is divided into Stage 1.

[0079] After determining the subgraph to which node 1 belongs, the compiler can update the in-degree of node 1's successor nodes (node ​​2 and node 4). Specifically, node 2 and node 4 are respectively subtracted by 1, and the in-degree of node 2 and node 4 are updated to 0, which are recorded as current nodes. At this time, the compiler can respectively determine whether node 2 and node 4 use the same resources of the computing device as their predecessor node 1. When node 2 and node 1 use the same resources of the computing device, node 1 and node 2 are divided into the same subgraph, that is, node 2 is divided into the subgraph (Stage 1) to which node 1 belongs. When node 4 and node 1 use different resources of the computing device, node 1 and node 4 are divided into different subgraphs. For example, the compiler can divide node 4 into a separate subgraph, such as Stage 2.

[0080] After determining the subgraph to which nodes 2 and 4 belong, the compiler can update the in-degree of the remaining nodes in the directed acyclic graph. Specifically, the compiler can update the in-degree of the successor nodes (nodes 3 and 5) of nodes 2 and 4 according to the dependency relationship between the nodes in the directed acyclic graph, and subtract 1 from the in-degree of the successor node (node ​​3) of node 2. At this time, the in-degree of node 3 is updated to 0 and recorded as the current node. At the same time, the compiler can subtract 1 from the in-degree of the successor node (node ​​5) of node 4. At this time, the in-degree of node 5 is updated to 1.

[0081] The predecessor node of node 3 is node 2, and node 3 and node 2 use different resources of the computing device. The compiler can divide node 3 and node 2 into different subgraphs, that is, divide node 3 into a separate subgraph, such as Stage 3.

[0082] After determining the subgraphs to which nodes 3 and 4 belong, the compiler can continue to update the in-degree of node 5, subtract 1 from the in-degree of node 5, and update the in-degree of node 5 to 0, which is recorded as the current node. At this point, the compiler can partition the current node 5 into a separate subgraph (for example, into subgraph Stage 4).

[0083] For example, Figure 8As shown, in order to reduce the number of subgraphs, the compiler can also divide the current nodes 3 and 4 into the same subgraph when there is no data dependency between nodes 3 and 4. Specifically, when node 2 and node 1 use the same resources of the computing device, the compiler divides node 1 and node 2 into the same subgraph, that is, divides node 2 into the subgraph (Stage 1) to which node 1 belongs. At the same time, the compiler can determine that node 4 should be divided into a subgraph different from node 1, but the subgraph to which node 4 belongs is not determined at this time.

[0084] After determining the subgraph to which node 2 belongs, the compiler can update the in-degree of node 2's successor node (node ​​3) and update the in-degree of node 3 to 0. At this time, since both nodes 3 and 4 are nodes with in-degree 0 and there is no data dependency between nodes 3 and 4, the compiler can divide nodes 3 and 4 into the same subgraph, as in Stage 2.

[0085] After determining the subgraphs to which nodes 3 and 4 belong, the compiler can further update the in-degree of node 5 to 0, and node 5 is now the current node. Since node 5 and its predecessor nodes (nodes 3 and 4) use different resources of the computing device, the compiler divides node 5 into a separate subgraph, such as in Stage 3.

[0086] Furthermore, when the program needs to repeatedly execute the above directed acyclic graph, the compiler can determine the software pipeline of the program according to the above pipeline level division. For example, the program includes 3 loop iterations, and the above directed acyclic graph is repeated 3 times. The compiler can automatically generate the software pipeline corresponding to the program according to the above determined pipeline levels. Figure 8 Taking the pipeline division of as an example, the compiler can automatically generate Fig. 9 Thus, when the computing device executes the program, different resources of the computing device can perform different operations of the program in parallel, thereby improving the parallelism and computing efficiency of the program.

[0087] In another embodiment, the disclosed embodiment can also perform subgraph partitioning according to the topological sorting of the directed acyclic graph, and select the optimal subgraph partitioning method. Fig.10 As shown, the above method may include:

[0088] S1001. Obtain a directed acyclic graph corresponding to the program, wherein the directed acyclic graph includes a plurality of nodes; wherein the directed acyclic graph corresponding to the program may be as follows: Figure 7 or Figure 8 The directed acyclic graph shown.

[0089] S1002, determining a topological sorting of nodes according to data dependencies of the nodes;

[0090] Among them, topological sorting refers to arranging all the nodes in the directed acyclic graph into a linear sequence. Specifically, all the nodes can be arranged into a linear sequence according to the data dependency relationship of each node in the directed acyclic graph, so that the successor node of any node in the graph is arranged after the node to ensure the correctness of the operation.

[0091] S1003: Divide the directed acyclic graph into at least one subgraph according to the topological order of the nodes and the resources used by the nodes, wherein each subgraph may include at least one node, and each subgraph may serve as a pipeline stage, so that the compiler may determine the software pipeline of the program according to the pipeline stage.

[0092] Specifically, if consecutive nodes in the topological sorting use the same resources, the consecutive nodes can be divided into the same subgraph. If adjacent nodes in the topological sorting use different resources, the two adjacent nodes can be divided into different subgraphs. Furthermore, if there is no data dependency between adjacent nodes and they use different resources, the adjacent nodes can be integrated into the same subgraph to reduce the number of subgraphs, thereby improving the processing efficiency of the computing device by reducing the number of pipeline stages.

[0093] Optionally, there may be multiple topological orderings corresponding to the directed acyclic graph. The compiler may also obtain all possible topological orderings of the directed acyclic graph, and implement subgraph partitioning based on each topological ordering. Each topological ordering may correspondingly obtain a subgraph partitioning scheme.

[0094] Furthermore, based on the subgraph partitioning schemes obtained by the above-mentioned multiple topological sortings, the compiler is also used to:

[0095] S1004, select the optimal subgraph partitioning scheme, and determine the software pipeline of the program according to the optimal subgraph partitioning scheme. For example, the compiler can select the subgraph partitioning scheme with the least number of subgraphs from multiple subgraph partitioning schemes as the optimal subgraph partitioning scheme. For another example, the compiler can obtain the above-mentioned various subgraph partitioning schemes to determine the software pipeline of the program, and obtain the running time of the program running according to the above-mentioned software pipeline. The compiler can use the subgraph partitioning scheme corresponding to the pipeline with the shortest running time as the optimal subgraph partitioning scheme.

[0096] The embodiment is described below in conjunction with the accompanying drawings. Figure 8The directed acyclic graph shown in FIG. The directed acyclic graph includes nodes 1, 2, 3, 4 and 5, wherein nodes 1 and 2 use the same resources, for example, nodes 1 and 2 use the memory access resources of the computing device to implement the data loading operation; nodes 3, 4 and 5 use different resources respectively, for example, node 3 uses the cluster 2 of the computing device to perform a computing operation, node 4 uses the cluster 3 of the computing device to perform another computing operation, and node 5 uses the memory access resources of the computing device to implement the operation of saving the calculation result.

[0097] Based on the data dependency between nodes 1 and 5, the compiler can obtain the following topological sorting:

[0098] Topological sorting 1: node 1-node 2-node 3-node 4-node 5;

[0099] Topological sorting 2: node 1-node 2-node 4-node 3-node 5;

[0100] Topological sort 3: node 1 - node 4 - node 2 - node 3 - node 5.

[0101] Based on the above topological sorting 1, since the adjacent nodes 1 and 2 use the same resources of the computing device, the above nodes 1 and 2 can be divided into the same subgraph. Node 2 and node 3 use different resources of the computing device, so node 3 is divided into a different subgraph. Node 3 and node 4 use different resources of the computing device, so node 4 and node 3 can be divided into different subgraphs. Node 4 and node 5 use different resources of the computing device, so node 4 and node 5 can be divided into different subgraphs, that is, node 3, node 4 and node 5 are respectively divided into separate subgraphs. Further, since there is no data dependency relationship between node 3 and node 4, node 3 and node 4 can be integrated into the same subgraph.

[0102] Similarly, based on the above topological sorting 2, nodes 1 and 2 can be divided into the same subgraph. Nodes 4 and 3 can be divided into the same subgraph. Node 5 is divided into a separate subgraph.

[0103] Based on the above topological sorting 3: Node 1 and Node 4 use different resources of the computing device, so Node 1 and Node 4 are divided into different subgraphs, such as subgraphs Figure 1 Kazuko Figure 2 Node 4 and node 2 use different resources of the computing device, so node 2 is divided into a separate subgraph (e.g., sub Figure 3 ). Node 3 and node 2 use different resources of the computing device, so node 3 is divided into a separate subgraph (e.g., sub Figure 4). Node 5 and node 3 use different resources of the computing device, so node 3 is divided into a separate subgraph (e.g., sub Figure 5 Furthermore, since there is no data dependency between node 2 and node 4, node 2 and node 4 can be integrated into the same subgraph.

[0104] Based on the three subgraph partitioning schemes obtained from the above three topological sortings, the compiler can select the optimal subgraph partitioning. For example, the compiler can select the subgraph partitioning scheme with the least number of subgraphs and use the subgraph partitioning scheme as the optimal subgraph partitioning scheme. Continuing with the above example, the subgraph partitioning schemes determined based on topological sorting 1 and topological sorting 2 are the same, and the number of subgraphs in the subgraph partitioning scheme is less than the number of subgraphs in the subgraph partitioning scheme obtained based on topological sorting 3. Therefore, the subgraph partitioning scheme determined based on topological sorting 1 and topological sorting 2 can be used as the optimal subgraph partitioning scheme. Afterwards, the compiler can determine the software pipeline of the program based on the subgraph partitioning scheme to generate an executable file that can be executed by the computing device based on the software pipeline.

[0105] In one embodiment, the operations in the program may also include some operations with low hardware execution cost, such as scalar operations, etc. The optimization benefits of such operations entering the software pipeline and executing them in parallel with other operations are low. Therefore, such operations may not participate in the above-mentioned division of pipeline stages and generation of software pipelines.

[0106] Optionally, the calculation operations in the program may also involve scalar operations, which are calculated using the scalar calculation unit of the computing device. However, the scalar operations and tensor operations of the computing device are usually executed sequentially by default, and there is no opportunity for scalar operations and other tensor operations to be performed in parallel. Moreover, since scalar operations take less time than other vector or matrix operations, even if scalar operations and tensor operations are performed in parallel, the optimization benefits obtained are low. Therefore, in order to reduce the number of subgraphs in the directed acyclic graph, the compiler of the present disclosure can also divide the nodes involving scalar operations in the directed acyclic graph into arbitrary subgraphs while ensuring that the data dependencies are correct, and no longer divide the pipeline stages separately for scalar operations. Optionally, if the current node involves a scalar operation, the compiler divides the current node into the subgraph to which the predecessor node of the current node belongs.

[0107] Of course, in other embodiments, if the computing device supports parallel execution of scalar operations and tensor operations, the compiler may also treat the nodes involving scalar operations in the directed acyclic graph as separate pipeline stages to execute them in parallel with other tensor operations.

[0108] Continue to see Figure 7 or Figure 8In the directed acyclic graph shown, if the resources used by nodes 1, 2, and 5 are memory access resources of the computing device, node 3 uses the tensor computing unit (cluster 2) of the computing device to perform tensor operations, and node 4 involves scalar operations, and the resources used are the scalar computing units of the computing device. In one embodiment, the compiler may divide the above-mentioned nodes 1, 2, and 4 into the same subgraph, and divide nodes 3 and 5 into separate subgraphs respectively. In another embodiment, the compiler may divide the above-mentioned nodes 1 and 2 into the same subgraph, divide nodes 3 and 4 into the same subgraph, and divide node 5 into a separate subgraph. In yet another embodiment, the compiler may also divide nodes 1 and 2 into the same subgraph, divide node 3 into a separate subgraph, and divide nodes 4 and 5 into the same subgraph.

[0109] In one embodiment, the nodes in the directed acyclic graph include data transfer operations within the computing device, or the computing operations in the directed acyclic graph may introduce some data transfer operations. For example, there will be data interaction between computing nodes that have data dependencies and use different resources. In this way, when using different resources of the computing device to execute a program, it is necessary to perform data transfer operations. The disclosed embodiment records this type of data transfer operation used to implement data interaction within the computing device as a MOVE operation. The MOVE operation is used to implement data interaction between different clusters of computing devices, or data interaction between different processor cores in the same cluster. The resources used by the MOVE operation are memory access resources on the computing device, such as CDMA, MVDMA, rather than computing resources. Optionally, the MOVE operation can also participate in the division of the above-mentioned pipeline stages and the generation of software pipelines. For example, the compiler can treat the MOVE operation as a separate pipeline stage based on the memory access resources of the computing device used by the MOVE operation.

[0110] Return to see Figure 2 and Figure 3 As described above, the storage resources of the hardware device may include external storage resources (such as Figure 2 external storage device 240), first-level storage resources (such as storage modules inside the processor core) and second-level storage resources (such as Figure 3 The storage core 330 of the computing device is used to implement data exchange between storage resources at all levels. The following examples illustrate the operations that the computing device needs to complete when performing a task, as well as the resources required for each operation:

[0111] Load operation (Load): used to transfer data from external storage resources to second-level storage resources; the resource required for the load operation may be the memory access resource GDMA of the computing device.

[0112] MOVE operation: used to move data from the second-level storage resources to the first-level storage resources; the resources required for the MOVE operation may be the memory access resource MVDMA of the computing device.

[0113] Computation operation: used to perform calculations using data in the first-level storage resources to obtain calculation results; the resources used by the calculation operation are the computing resources of the computing device (such as at least one processor core).

[0114] MOVE operation: used to write the calculation results in the first-level storage resources back to the second-level storage resources; the resources used by the MOVE operation can be the memory access resource MVDMA of the computing device.

[0115] Save operation: used to write the calculation results in the second-level storage resource back to the external storage resource; the resource used by the save operation may be the memory access resource GDMA of the computing device.

[0116] The compiler can determine that there is a data dependency between the above operations, and generate a directed acyclic graph based on the data dependency between the operations. As shown in the above example, the operations need to be executed sequentially, and the operations with data dependencies use different resources of the computing device, so each of the above operations forms a separate pipeline stage. The specific division method of the subgraph in the program can be found in the previous text, so the program containing the above operations will contain at least 5 pipeline stages.

[0117] In the above embodiment, if the MOVE operation is also added to the software pipeline, it will lead to too many stages of the software pipeline. Therefore, in order to optimize the number of stages of the software pipeline, the embodiment of the present disclosure can also set the resources used by the above MOVE operation to specific computing resources.

[0118] Continue to see Figure 3 , each cluster includes a storage core, and each processor core includes a storage module (not shown in the figure). Therefore, if the MOVE operation accesses the corresponding storage resource in the computing resource, it can be determined that the MOVE operation uses the corresponding computing resource. Among them, the MOVE operation accessing the corresponding storage resource in the computing resource means that the MOVE operation performs a read or write operation on the corresponding storage resource in the computing resource. Specifically, the MOVE operation is to write data to the storage resource inside the computing resource. For example, the MOVE operation is to write data to the storage module inside the processor core or read data from the storage module inside the processor core. Then the resources used by the MOVE operation can be set to the corresponding processor core. For another example, the MOVE operation is to write data to the storage core of the cluster or read data from the storage core of the cluster. Then the resources used by the MOVE operation can be set to the corresponding cluster.

[0119] Furthermore, the compiler can mark the computing resources used by the MOVE operation in the directed acyclic graph to divide the program into pipeline levels. Furthermore, the compiler can wrap the MOVE operation and its related computing operations with a region and mark them with a hint. The region refers to a portion of a directed acyclic graph that has only a single entry node.

[0120] Continuing with the above example, the disclosed embodiment can set the resources used by the above two MOVE operations as the resources used by the computing operation. The two MOVE operations and the computing operation can be equivalent to a fused computing operation, so that the pipeline stages of the above program can be reduced to: load operation, fused computing operation and save operation. Compared with the above embodiment, the disclosed embodiment reduces the number of pipeline stages. Furthermore, the fused computing operation can be wrapped with a region and marked with a hint so that the compiler can correctly generate instructions that can be executed by the computing device when generating an executable file according to the software pipeline.

[0121] In one embodiment, the directed graph determined according to the data dependency of the program may also contain a dependency cycle, and the compiler may convert the cyclic graph into a directed acyclic graph. Specifically, the method further includes:

[0122] If the directed graph corresponding to the program contains a dependency cycle, a strongly connected component in the directed graph is determined, and the strongly connected component is replaced with a single new node to obtain a directed acyclic graph corresponding to the program.

[0123] In a directed graph, if any two nodes a and b are reachable from each other (i.e., there is a path a->b and a path b->a), then these nodes constitute a connected component. A strongly connected component (SCC) is a maximal connected quantity. For a connected component, if every maximal connected component that contains it is itself, it is called a strongly connected component.

[0124] In the disclosed embodiment, if the directed graph corresponding to the program contains a dependency cycle, the compiler can determine the strongly connected components in the directed graph and replace the strongly connected components in the directed graph with equivalent new nodes to obtain the directed acyclic graph corresponding to the program. Furthermore, the new nodes corresponding to the strongly connected components may not participate in the above-mentioned pipeline level division and software pipeline generation, and the strongly connected components can be executed separately.

[0125] The present disclosure also provides a program optimization device, which may include a memory and a processor, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the method steps of any of the above embodiments are implemented. Specifically, when the computer executable program is executed by the processor, the following method steps are implemented:

[0126] Obtaining a directed acyclic graph corresponding to the program, wherein the directed acyclic graph includes a plurality of nodes;

[0127] Dividing the directed acyclic graph into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes;

[0128] Each subgraph serves as a pipeline stage, and the pipeline stage is used to determine the pipeline of the program operation, so that multiple resources of the computing device can run different pipeline stages in the program in parallel according to the pipeline.

[0129] Among them, the working principle of the program optimization device is basically consistent with the execution process of the method steps in the above embodiment, and the details can be found in the above description.

[0130] An embodiment of the present disclosure further provides a computer program product, including a computer program or instructions. Optionally, the computer program product may be a compiler. When the instructions or program of the computer program product are executed by a processor, the method steps of any of the above embodiments may be implemented.

[0131] The embodiments of the present disclosure further provide a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by at least one processor, the method steps of any of the above embodiments can be implemented.

[0132] It should be clear that the working principle of the computer program product and the computer program stored in the storage medium is basically consistent with the execution process of the method steps in the above embodiment, and please refer to the above description for details.

[0133] Among them, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM) and other memories; it can also be various devices including one or any combination of the above memories.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-mentioned embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the patent of the present disclosure shall be subject to the attached claims.

Claims

1. A program optimization method, characterized in that: The method comprises: Obtaining a directed acyclic graph corresponding to the program, wherein the directed acyclic graph includes a plurality of nodes; Dividing the directed acyclic graph into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes; Each subgraph serves as a pipeline stage, and the pipeline stage is used to determine the pipeline of the program operation, so that different resources of the computing device can run different pipeline stages in the program in parallel according to the pipeline.

2. The method according to claim 1, characterized in that The step of dividing the directed acyclic graph into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes comprises: Determining, according to the data dependency relationship of the nodes, whether the current node and a predecessor node of the current node use the same resources; If the current node and the predecessor node of the current node use the same resources, the current node is divided into the subgraph to which the predecessor node of the current node belongs.

3. The method according to claim 2, characterized in that The step of dividing the directed acyclic graph into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes further includes: If the current node and the predecessor node of the current node use different resources, the current node and the predecessor node of the current node are divided into different subgraphs.

4. The method according to claim 3, characterized in that The number of the current nodes is multiple; the directed acyclic graph is divided into at least one subgraph according to the data dependency relationship of the nodes and the resources used by the nodes, and further includes: If there is no data dependency relationship between the multiple current nodes, the multiple current nodes are divided into the same subgraph.

5. The method according to any one of claims 2 to 4, characterized in that: The current node is a source node with an in-degree of 0.

6. The method according to claim 5, characterized in that The method further comprises: After determining the subgraph to which the current node belongs, the in-degree of the successor node of the current node is updated to update the current node.

7. The method according to claim 1, characterized in that Dividing the directed acyclic graph into at least one subgraph according to the data dependency of the nodes and the resources used by the nodes, including: Determine a topological sorting of the nodes according to data dependencies of the nodes; The directed acyclic graph is divided into at least one subgraph according to the topological ordering of the nodes and the resources used by the nodes.

8. The method according to claim 1, characterized in that The method further comprises: If the current node involves a scalar operation, the current node is divided into a subgraph to which a predecessor node of the current node belongs.

9. The method according to claim 1, characterized in that: The method further comprises: If the current node involves a data transfer operation within a computing device, the resources used by the data transfer operation within the computing device are set to specific computing resources.

10. The method according to claim 1, characterized in that The method further comprises: If the directed graph corresponding to the program contains a dependency cycle, a strongly connected component in the directed graph is determined, and the strongly connected component is replaced with a single new node to obtain a directed acyclic graph corresponding to the program.

11. The method according to claim 1, characterized in that: The method further comprises: A pipeline for executing the program is determined according to at least one pipeline stage.

12. A program optimization device, characterized in that: The device includes a memory and a processor, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the program optimization method according to any one of claims 1 to 11 is implemented.

13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the program optimization method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, wherein: The computer storage medium is used to store computer programs or instructions, and when the computer programs or instructions are executed by the processor, the program optimization method according to any one of claims 1 to 11 is implemented.

Images