Compilation method, device, equipment and storage medium based on heterogeneous system

Abstract

The present invention discloses a compilation method, apparatus, device and storage medium based on a heterogeneous system. The heterogeneous system includes multiple computing cores with different architectures. The method includes: obtaining a computational graph corresponding to a source language code, the source language code is written based on a programming model, and the programming model is algorithmically described by a scalar programming language and a tensor programming language; dividing the computational graph into at least one target graph area, the at least one target graph area including a scalar graph area containing scalar computing nodes and / or a tensor graph area containing tensor computing nodes; generating a corresponding binary instruction code segment for each target graph area; based on the dependency relationship between the target graph areas and the binary instruction code segment, generating a binary instruction sequence corresponding to the source language code. In the present disclosure, when compiling and developing based on a heterogeneous system, the source language code can use a scalar programming language and a tensor programming language to describe the algorithm, thereby improving development efficiency.

Description

Technical Field

[0001] The present disclosure relates to, but is not limited to, the field of compilation development, and in particular to a compilation method, apparatus, device, and storage medium based on heterogeneous systems. Background Art

[0002] With the rise of computing-intensive fields such as artificial intelligence, high-performance data analysis, and financial analysis, traditional general-purpose computing methods can no longer meet our demand for computing power. Therefore, a more powerful computing method, namely heterogeneous computing, has been proposed.

[0003] Heterogeneous computing is a special type of parallel distributed computing system, primarily referring to a computing approach that uses computing units with different instruction sets and architectures to form a system. In a heterogeneous computing system, common heterogeneous systems (also known as heterogeneous devices) can include a combination of different computing cores, such as central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), tensor processing units (TPUs), and application-specific integrated circuits (ASICs).

[0004] In heterogeneous systems, since each computing core usually has its own instruction set and programming model, when developing programs based on heterogeneous systems, it is necessary to program and compile the programming model of each computing core separately, resulting in low development efficiency. Summary of the Invention

[0005] The present disclosure provides a compilation method, apparatus, device, and storage medium based on heterogeneous systems to achieve unified compilation of heterogeneous systems and improve development efficiency.

[0006] According to the first aspect of the embodiment of the present disclosure, the present disclosure provides a compilation method based on a heterogeneous system, wherein the heterogeneous system includes multiple computing cores with different architectures. The method includes: obtaining a computational graph corresponding to a source language code, the source language code is written based on a programming model, and the programming model is algorithmically described by a scalar programming language and a tensor programming language; dividing the computational graph into at least one target graph area, at least one target graph area including a scalar graph area containing scalar computing nodes and / or a tensor graph area containing tensor computing nodes; generating a corresponding binary instruction code segment for each target graph area, the binary code segment corresponding to the scalar graph area can be executed in a first computing core for scalar computing, and the binary code segment corresponding to the tensor graph area can be executed in a second computing core for tensor computing; based on the dependency relationship between the target graph areas and the binary instruction code segments, generating a binary instruction sequence corresponding to the source language code.

[0007] In the above solution, the programming model includes a built-in function interface written in a tensor programming language that can be run in the second computing core, and the built-in function interface is used to be called to perform tensor calculations on tensor data.

[0008] In the above scheme, obtaining a computational graph corresponding to the source language code includes: converting the source language code into a first intermediate representation, the first intermediate representation being used to represent scalar data and / or tensor data; generating a computational graph based on the first intermediate representation, wherein the computational nodes in the computational graph represent functions corresponding to the source language code, and the directed edges in the computational graph represent dependencies between functions.

[0009] In the above scheme, after generating a computational graph based on the first intermediate representation, the method further includes: optimizing the computational graph; wherein the optimization includes at least one of the following processes: deleting redundant computational nodes, merging computational nodes, equivalent conversion of complex computations to simple computations, and merging loop blocks.

[0010] In the above scheme, the computation graph is divided into at least one target graph area, including: dividing the computation nodes in the computation graph that have a dependency relationship and the same computation type into a computation area; counting the mappable computation cores of each computation node in each computation area; dividing the computation nodes in each computation area that have a dependency relationship and the same mappable computation core into a graph area; and determining at least one target graph area based on the graph area in each computation area.

[0011] In the above solution, the calculation area is a scalar calculation area; and determining at least one target graph area according to the graph area in each calculation area includes: determining the graph area as the at least one target graph area.

[0012] In the above scheme, the computing area is a tensor computing area; according to the graph area in each computing area, at least one target graph area is determined, including: creating a graph area set for the i-th graph area in the graph area; traversing the graph area set, dividing the computing nodes in the i-th graph area that have a dependency relationship and can be mapped to the same computing core into a candidate graph area, where i is a positive integer; determining the first benefit value of the i-th graph area according to the execution time of the candidate graph area of ​​the i-th graph area on the corresponding computing core; determining a second benefit value that is greater than the preset benefit value and has the largest value from the first benefit value; updating the candidate graph area corresponding to the second benefit value to the graph area set, and returning to the step of traversing the graph area set until the first benefit values ​​are all less than or equal to the preset benefit value; determining the candidate graph area corresponding to the first benefit value as the target graph area.

[0013] In the above scheme, the computation graph is divided into at least one target graph area, including: dividing the computation nodes of the same type that have dependencies in the computation graph into one computation area; calculating the execution time of each computation node in each computation area in the mappable computation core, and determining the mappable computation core with the smallest execution time as the target computation core of the computation node; merging the computation nodes with the same target computation core in each computation area into one graph area; and determining at least one target graph area based on the merged graph area.

[0014] In the above scheme, a corresponding binary instruction code segment is generated for each target graph area, including: determining a mapping relationship between each target graph area in at least one target graph area and a computing core; converting each target graph area into a second intermediate representation according to the mapping relationship; and generating a binary instruction code segment based on the second intermediate representation.

[0015] In the above solution, the first computing core includes at least one of the following computing cores: CPU, DSP; the second computing core includes at least one of the following computing cores: CPU, TPU, GPU, DSP, ASIC, FPGA.

[0016] According to a second aspect of an embodiment of the present disclosure, the present disclosure provides a compilation device for compiling programs for a heterogeneous system, wherein the heterogeneous system includes multiple computing cores with different architectures. The compilation device includes: a programming front-end module for obtaining a computational graph corresponding to a source language code, wherein the source language code is written based on a programming model, and the programming model is algorithmically described by a scalar programming language and a tensor programming language; a computational graph processing module for dividing the computational graph into at least one target graph area, wherein the at least one target graph area includes a scalar graph area containing scalar computing nodes and / or a tensor graph area containing tensor computing nodes; a computing core decoding module for generating a corresponding binary instruction code segment for each target graph area; and, based on the dependency relationship between the target graph areas and the binary instruction code segment, generating a binary instruction sequence corresponding to the source language code, wherein the binary code segment corresponding to the scalar graph area can be executed in a first computing core for scalar computing, and the binary code segment corresponding to the tensor graph area can be executed in a second computing core for tensor computing.

[0017] In the above solution, the programming model includes a built-in function interface written in a tensor programming language that can be run in the second computing core, and the built-in function interface is used to be called to perform tensor calculations on tensor data.

[0018] In the above scheme, a programming front-end module is used to convert the source language code into a first intermediate representation, which is used to represent scalar data and / or tensor data; a computational graph is generated based on the first intermediate representation, wherein the computational nodes in the computational graph represent functions corresponding to the source language code, and the directed edges in the computational graph represent dependencies between functions.

[0019] In the above scheme, the programming front-end module is also used to optimize the calculation graph; wherein the optimization processing includes at least one of the following processing: deleting redundant computing nodes, computing node fusion, equivalent conversion of complex calculations to simple calculations, and loop block merging.

[0020] In the above scheme, the computational graph processing module is used to divide the computational nodes in the computational graph that have dependencies and the same computational type into a computational area; count the mappable computational cores of each computational node in each computational area; divide the computational nodes in each computational area that have dependencies and the same mappable computational core into a graph area; and determine at least one target graph area based on the graph area in each computational area.

[0021] In the above solution, the calculation area is a scalar calculation area; the calculation graph processing module is used to determine at least one graph area corresponding to the calculation graph as at least one target graph area.

[0022] In the above scheme, the calculation area is a tensor calculation area; the calculation graph processing module is used to create a graph area set for the i-th graph area in the graph area; traverse the graph area set, and divide the computing nodes in the i-th graph area that have dependencies and can be mapped to the same computing core into a candidate graph area, where i is a positive integer; determine the first benefit value of the i-th graph area based on the execution time of the candidate graph area of ​​the i-th graph area on the corresponding computing core; determine the second benefit value that is greater than the preset benefit value and has the largest value from the first benefit value; update the candidate graph area corresponding to the second benefit value to the graph area set, and return to the step of traversing the graph area set until the first benefit values ​​are all less than or equal to the preset benefit value; determine the candidate graph area corresponding to the first benefit value as the target graph area.

[0023] In the above scheme, the computational graph processing module is also used to divide the computational nodes of the same type that have dependencies in the computational graph into a computational area; calculate the execution time of each computational node in each computational area in the mappable computational core, and determine the mappable computational core with the shortest execution time as the target computational core of the computational node; merge the computational nodes with the same target computational core in each computational area into a graph area; and determine at least one target graph area based on the merged graph areas.

[0024] In the above scheme, the computing core decoding module is used to determine the mapping relationship between each target graph area in at least one target graph area and the computing core; according to the mapping relationship, each target graph area is converted into a second intermediate representation; based on the second intermediate representation, a binary instruction code segment is generated.

[0025] According to a third aspect of an embodiment of the present disclosure, the present disclosure provides a compilation device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: when executing the executable instructions, implement the method described in the first aspect and any of the solutions thereof.

[0026] According to a fourth aspect of an embodiment of the present disclosure, the present disclosure provides a computer-readable storage medium, which stores an executable program, wherein when the executable program is executed by a processor, it implements the method described in the first aspect and any of the solutions thereof.

[0027] In this disclosure, when developing programs based on heterogeneous systems, since the source language code can describe the algorithm using both scalar and tensor programming languages, there is no need to program separately for different heterogeneous computing cores. Instead, scalar and tensor calculations can be written simultaneously during the programming process, greatly improving development efficiency. Furthermore, through computational graph partitioning, the same computational function can be mapped to different heterogeneous computing cores for execution, shortening the execution time of the function and improving the effective utilization of computing resources.

[0028] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0030] Figure 1 Schematic diagram of the structure of a heterogeneous system in an embodiment of the present disclosure;

[0031] Figure 2 is a schematic diagram of the structure of a compiler in an embodiment of the present disclosure;

[0032] Figure 3 Schematic diagram of an implementation flow of a compilation method in an embodiment of the present disclosure;

[0033] Figure 4 A schematic diagram describing a source language code in an embodiment of the present disclosure;

[0034] Figure 5 is a schematic diagram of a calculation graph in an embodiment of the present disclosure;

[0035] Figure 6 Schematic diagram of an implementation process of partitioning a computation graph in an embodiment of the present disclosure;

[0036] Figure 7 Schematic diagram of another implementation process of partitioning a computation graph in an embodiment of the present disclosure;

[0037] Figure 8 is a schematic diagram of a tensor calculation area in an embodiment of the present disclosure;

[0038] Figure 9 Schematic diagram of the structure of a compilation device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0040] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present disclosure. The singular forms "a," "an," and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms "first," "second," "third," etc. may be used to describe various information in the embodiments of the present disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present disclosure, "first information" may also be referred to as "second information," and similarly, "second information" may also be referred to as "first information." Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0042] With the rise of computing-intensive fields such as artificial intelligence, high-performance data analysis, and financial analysis, traditional general-purpose computing methods can no longer meet our demand for computing power. Therefore, a more powerful computing method, namely heterogeneous computing, has been proposed.

[0043] Heterogeneous computing mainly refers to a computing method that uses computing units with different types of instruction sets and architectures to form a system. In a heterogeneous computing system, common categories of heterogeneous computing cores may include CPU, GPU, DSP, FPGA, TPU, ASIC, etc. For example, a heterogeneous system may integrate a general-purpose computer such as a CPU and GPU, a system on chip (SoC) that integrates CPU, GPU and DSP chips, a machine learning system that integrates CPU and TPU / ASIC / FPGA / ASIC, etc. In a heterogeneous system, since each computing core usually has its own instruction set and programming model, when developing a program based on a heterogeneous system, it is necessary to program and compile separately for the programming model of each computing core, resulting in low development efficiency.

[0044] Furthermore, for the same computing task (e.g., a computing function), the core computing program is executed only on one or one type of computing core in a heterogeneous system, and the same core computing program cannot run simultaneously across multiple computing cores (this can be understood as the same core computing program can only run on one or one type of computing core, and cannot run in parallel on different types of computing cores). Therefore, although multiple computing tasks can be executed in parallel, heterogeneous parallel operation cannot be achieved for a single computing task, resulting in low utilization of computing resources.

[0045] Therefore, in order to solve the above problems, an embodiment of the present disclosure provides a compilation method, which can be applied to a heterogeneous system. Figure 1 This is a schematic diagram of the structure of a heterogeneous system in the embodiment of the present disclosure, see Figure 1 As shown by the solid line in the middle, the heterogeneous system 10 may include a programming program 11 (also referred to as a development program or development software) of the heterogeneous system 10 and a heterogeneous computing core 12. Optionally, the heterogeneous computing core 12 may include multiple computing cores with different architectures. These computing cores include one or more computing cores A 121 (such as a first computing core) for scalar calculations and one or more computing cores B 122 (such as a second computing core) for tensor calculations. Here, when there are multiple computing cores A, the multiple computing cores A are computing cores of the same architecture; similarly, when there are multiple computing cores B, the multiple computing cores B are computing cores of the same architecture. Exemplarily, computing core A can be a CPU, DSP, etc., and computing core B can be a CPU, GPU, TPU, FPGA, DSP, AISC, etc.

[0046] Still see Figure 1 As shown, the programming program 11 may include a programming model module 111 and a compiler 112. The programming model module 111 is used to describe the user-written source language code 20 that is independent of the underlying heterogeneous computing core 12. The compiler 112 is used to compile the source language code 20 based on the heterogeneous computing core to obtain a corresponding binary instruction sequence.

[0047] In some possible implementations, in order to enable unified programming of heterogeneous systems, the programming program 11 may support general applications (e.g., including scalar calculation functions) and may also support artificial intelligence (AI) deep learning applications (e.g., including tensor calculation functions). Of course, the programming program 11 may also support applications of other frameworks, which is not specifically limited in the present disclosure.

[0048] In some possible implementations, Figure 2 This is a schematic diagram of the structure of a compiler in the embodiment of the present disclosure, see Figure 2 As shown, the compiler 112 may include a programming front-end module 1121 , a computational graph processing module 1122 , and a computational kernel compilation module 1123 .

[0049] In some possible implementations, the compiler 112 may adopt a three-layer structure, with the first layer being a programming front-end module 1121. The first layer may include a programming language front-end analyzer 1121a and a deep learning reasoning graph front-end analyzer 1121b, wherein the programming language front-end analyzer is used to parse the source language code of general applications, and the deep learning reasoning graph front-end analyzer is used to parse neural network graphs. The second layer is a computational graph processing module 1122, which may include a graph optimizer 1122a and a task allocator 1122b for multiple heterogeneous computing cores. The third layer is a computing core compilation module 1123, which may include compilers for each computing core, such as computing core compiler A 1123a for computing core A and computing core compiler B 1123b for computing core B. The compilers for each computing core receive tasks issued by the second layer, generate binary instruction code segments corresponding to each computing core, and then generate binary instruction sequences corresponding to the source language code.

[0050] The compilation method provided by the embodiment of the present disclosure is described below in conjunction with the above-mentioned heterogeneous system.

[0051] Figure 3 This is a schematic diagram of an implementation flow of a compilation method in an embodiment of the present disclosure, see Figure 3 As shown, the above compilation method may include:

[0052] S301: The programming model module obtains source language code written by a user using a programming language according to a programming model.

[0053] The above programming model is algorithmically described by scalar programming language and tensor programming language.

[0054] In practical applications, the programming model may include a scalar computing function written in a scalar programming language that can be performed in computing core A, and also includes a tensor computing function written in a tensor programming language that can be performed in computing core B (the tensor computing function can be understood as a built-in function in the programming model). Optionally, in order to use the tensor computing function to perform tensor calculations, the programming model may also include a built-in function interface (i.e., a tensor computing function interface), which is used to call the tensor computing function to perform tensor calculations on the input tensor data. Exemplarily, the above-mentioned built-in function interface is obtained by expansion based on a scalar programming language (such as C / C++ language).

[0055] In some possible implementations, for building built-in function interfaces, CPUs, GPUs, DSPs, and TPUs can use their underlying instruction sets to achieve their highest performance. ASICs can implement their own functions. FPGAs can use the Verilog hardware description language.

[0056] In the embodiment of the present disclosure, since the programming model includes both scalar computing functions and tensor computing functions, the differences between heterogeneous computing cores can be shielded. Even if the heterogeneous computing cores change (such as expanding new scalar computing cores and / or tensor computing cores), the source language code written according to the above programming model can remain unchanged.

[0057] For example, Figure 4 This is a schematic diagram describing a source language code in an embodiment of the present disclosure, see Figure 4 As shown in the above source code, L2LossKernel is the function name of this source code, and the box above the function is the built-in function interface. Among them, scalar data can be considered as a 0-dimensional vector data, corresponding to Figure 4 Tensor data can be considered as vector data with a dimension greater than or equal to 1, corresponding to oklang::tensor in the figure. If the tensor data is 1-dimensional, it can be understood as a vector. If the tensor data is 2-dimensional, it can be understood as a matrix. If the tensor data is 3 or more dimensions, it can be understood as a multidimensional vector. Figure 4 The oklang::mul and oklang::muls in the __c__ are built-in functions (which can also be understood as tensor calculation functions). These built-in functions can be run in any computing core as long as the underlying heterogeneous computing core supports them.

[0058] S302, the programming front-end module generates a computational graph corresponding to the source language code.

[0059] As can be understood, the programming front-end module converts the source language code into a first intermediate representation capable of representing scalar data and / or tensor data. The programming front-end module then generates a computational graph corresponding to the source language code based on the first intermediate representation. The computational nodes in the computational graph represent functions corresponding to the source language code, and the directed edges in the computational graph represent dependencies between functions.

[0060] Exemplarily, the programming front-end module converts the first intermediate representation output by itself into a computational graph, which consists of nodes and directed edges connecting the nodes. Each node represents a function corresponding to the source language code (which can also be understood as a calculation or an operation). The function can be a tensor calculation function (such as a built-in function) or a scalar calculation function. A node used to represent a tensor calculation function can be called a tensor calculation node, and a node used to represent a scalar calculation function can be called a scalar calculation node. Figure 5 This is a schematic diagram of a calculation graph in an embodiment of the present disclosure, see Figure 5 As shown, directed edges From node u to node v, the directed edge Indicates that node v depends on node u.

[0061] Optionally, the programming model module passes the user-written source language code to the programming front-end module. If the user-written source language code is a neural network graph, the programming model module passes the source language code to the deep learning inference graph front-end analyzer.

[0062] In some possible embodiments, after S302 and before S303, the method may further include: a graph optimizer performing optimization processing on the computation graph generated by S302. Exemplarily, the optimization processing may include: deleting redundant computation nodes, merging computation nodes, converting complex computations to equivalent simple computations, merging loop blocks, etc. Of course, in actual applications, the optimization module may also perform other graph optimizations on the computation graph, which are not specifically limited in the present embodiment.

[0063] S303: The computation graph processing module divides the computation graph into at least one target graph area.

[0064] The at least one target graph region may include a scalar graph region including scalar computing nodes and / or a tensor graph region including tensor computing nodes.

[0065] Specifically, the above S303 can be implemented in the following two ways, but is not limited to the following ways.

[0066] The first way, Figure 6 This is a schematic diagram of an implementation process of partitioning a calculation graph in an embodiment of the present disclosure, see Figure 6 As shown, S303 may include:

[0067] S601: The task allocator divides computing nodes of the same type that have dependencies into a computing area.

[0068] For example, the task allocator can allocate at least one tensor computation node with a dependency relationship in the computation graph into a tensor computation region, and allocate at least one scalar computation node with a dependency relationship into a scalar computation region. In this way, the task allocator can obtain multiple computation regions corresponding to the computation graph.

[0069] S602: For each computing area, the task allocator counts the mappable computing cores of each computing node.

[0070] S603: The task allocator divides the computing nodes in each computing region that have dependencies and can map the same computing core into a graph region.

[0071] S604: The task allocator determines at least one target graph region of the computation graph based on at least one graph region in each computation region.

[0072] In some possible implementation manners, for a scalar graph region, the computable cores that can be mapped by the scalar computing nodes are one or more types of computing core A. If the heterogeneous computing cores only include one type of computing core A, such as a CPU, then the scalar graph region can be divided into one graph region; if the heterogeneous computing cores include two types of computing core A, such as a CPU and a DSP, then the scalar graph region can be divided into two graph regions, and so on. Exemplarily, the principle of graph region division is: the part with more parallel scalar computing nodes is divided into one graph region, and this block can be mapped to the DSP; other scalar computing nodes are divided into one or more graph regions, and these graph regions are mapped to the CPU. Further, the above S604 may include: the task allocator determines the graph regions of each computing region as target graph regions.

[0073] In some possible implementation manners, for a tensor graph region, the computable cores that can be mapped by each tensor computing node are one or more types of computing core B. The above S604 may include: for each tensor graph region, creating a graph region set for the i-th graph region (i is a positive integer, that is, any graph region of the computing region); traversing the graph region set, and dividing the computing nodes in the i-th graph region that have a dependency relationship and can be mapped to the same computing core into one candidate graph region; determining the first benefit value of the i-th graph region according to the execution duration of the candidate graph region of the i-th graph region on the corresponding computing core; determining the second benefit value that is greater than the preset benefit value and has the largest value from the first benefit values; updating the candidate graph region corresponding to the second benefit value to the graph region set, and returning to the step of traversing the graph region set, and iteratively calculating the benefit values when the i-th graph region is mapped to different computing cores until the first benefit values are all less than or equal to the preset benefit value; determining the candidate graph region corresponding to the first benefit value as the target graph region. Optionally, the preset benefit value can be 0.

[0074] In one embodiment, for a tensor graph region, step 1, the task allocator divides a tensor computing region into one graph region a0 that can be mapped to computing core B0, and creates a graph region set A for graph region a0. At this time, A = {a |0 ≤ i < m, m = 1}. Step 2, the task allocator traverses the graph regions in the graph region set A. When traversing to graph region a i , the tensor computing nodes in graph region a i that have a dependency relationship and can be mapped to other computing cores Bx (such as computing core B1, computing core B2,..., computing core Bn, n is a positive integer) are divided into one candidate graph region The candidate graph region can be a part of graph region a <00000 or the entire graph region a​i At this time, area a i It is equivalent to being split into multiple candidate regions, such as Figure area a i Different candidate graph regions can be mapped to different computational kernels. Mapped to the computational kernel Bx, candidate graph area Mappable computation kernel and graph region a i The computational core of the mapping is the same. Step 4, the task allocator determines the graph area a i Multiple candidate image regions after division The total execution time t in the corresponding mappable computation core i '. Step 5, the task allocator assigns the graph area a i The execution time t in the corresponding mappable computation kernel i The total execution time t of multiple candidate graph regions i 'Subtract, and you get the graph area a i The first benefit value p i =t i –t i ', and the first return value p i Add it to the benefit set P. Step 6: The task allocator selects a second benefit value p from the benefit set P that is greater than a preset benefit value (such as 0) and has the largest value. k And determine the payoff p k The corresponding area a k And area a k Candidate graph area Step 7: The task allocator deletes graph region a from the graph region set A. k , and the candidate graph area Add it to the graph region set A to update the graph region set A. Step 8: The task allocator repeats step 2 and retraverses the graph region set A until the first benefit values ​​in the benefit set P are all less than or equal to the preset benefit value. Step 9: The task allocator determines the candidate graph region in the graph region set A as the target graph region.

[0075] For example, Figure 8 This is a schematic diagram of a tensor calculation area in an embodiment of the present disclosure, see Figure 8 As shown in the figure, assume a tensor computation region is N0→N1→N2→N3→N4, where Ni (i=0, 1, 2, 3, 4) is a tensor computation node, and N0→N1 indicates that N1 depends on N0. Heterogeneous compute cores include three types of compute cores B (i.e., compute core B1, compute core B2, and compute core B3). N0, N1, N2, N3, and N4 can be mapped to compute core B1, N1 and N2 can be mapped to compute core B2, and N2, N3, and N4 can be mapped to compute core B3.

[0076] In the first step, the task allocator maps N0 to N4 to compute core B1. At this point, graph region a0 is N0 → N1 → N2 → N3 → N4, and the graph region set A = {a0}. The task allocator then determines the execution time t = 150 × T for graph region a0 in compute core B1.

[0077] In the second step, the task allocator traverses the graph region set A and divides N0 into a candidate graph region. Mapped in the computing core B1, N1→N2 is divided into a candidate graph area Mapped in the computing core B2 and divided N3→N4 into a candidate graph area Mapped in the computing core B1. At this time, the image area a0 is split into and Three candidate image regions, and determine The total execution time of the corresponding mappable computing core is t'=100×T (T is the clock cycle). The first benefit value p0 of the computation graph region a0 is calculated as t-t'=50×T.

[0078] In the third step, the task allocator divides N2→N3→N4 in the graph area a0 into a candidate graph area Mapped in the computing core B3, at this time, the image area a0 is divided into and Two candidate image regions, and determine The total execution time t'=120×T. Another first benefit value p0=t-t'=30×T is calculated for the region a0 of the graph.

[0079] At this time, the profit set P = {50,30}.

[0080] Step 4: The task allocator selects the candidate graph region corresponding to the second benefit value p0 = 50 × T Update the graph region set A. At this time, N1→N2 is mapped to B2, and N0 and N3→N4 are mapped to B1.

[0081] Step 5: The task allocator traverses the graph region set A again and assigns the graph region N1 in the image is divided into a candidate region Can be mapped to the computing core B2, dividing N2 into a candidate graph area Mapped in the computing core B1, the graph area Divided into Two candidate graph regions, and determine a first benefit value p1 = t-t' = -30 × T for graph region a0. And, the task allocator assigns graph region N1 is divided into a candidate graph area Mapped in the computing core B2, N2 is divided into a candidate graph area Mapped in the computing core B3, the graph area Divided into Two candidate graph regions are selected, and another profit value p1=t-t'=-20×T corresponding to the graph region a0 is determined.

[0082] Step 6: Task Allocator There is no other mappable computational kernel in N0. Divide into a candidate graph region Mapped in the computing core B3, the graph area Divided into A candidate graph area is selected, and another profit value p1=t-t'=20×T corresponding to the graph area a0 is determined.

[0083] After traversing the graph area set A, the profit set P = {-30, -20, 20}.

[0084] Step 7: The task allocator selects the candidate graph region corresponding to the second benefit value p1 = 20 × T Update the graph region set A. At this time, N1→N2 is mapped to B2, N0 is mapped to B1, and N3→N4 is mapped to B3.

[0085] In the eighth step, the task allocator traverses the graph region set A again, repeats the above steps, and obtains the benefit set P. At this time, all the first benefit values ​​in the benefit set P are less than or equal to 0. The task allocator terminates the program.

[0086] In the ninth step, the task allocator determines the updated graph area set A = {a0 = N1~N2, a1 = N0, a2 = N3~N4} as the target graph area, where N1~N2 are mapped to the computing core B2, N0 is mapped to the computing core B1, and N3~N4 are mapped to the computing core B3.

[0087] The second way, Figure 7 This is a schematic diagram of another implementation flow of partitioning the computation graph in the embodiment of the present disclosure, see Figure 7 As shown, S303 may include:

[0088] S701: The task allocator divides the computing nodes with dependencies and the same computing type in the computing graph into a computing area.

[0089] It is understandable that the specific execution process of S701 can refer to the relevant description of S601 and will not be repeated here.

[0090] S702 , the task allocator calculates the execution time of each computing node in each computing area in the mappable computing core.

[0091] It can be understood that the task allocator first treats each node in each computing area as a graph area, and uses integer linear programming algorithms, branch and bound algorithms, genetic algorithms, etc., with the goal of minimizing the execution time, and maps these blocks to each computing core to obtain the corresponding execution time.

[0092] S703 : The task allocator determines the mappable computing core with the shortest execution time as the target computing core of the computing node.

[0093] It can be understood that the task allocator determines the computing core with the shortest execution time among different mappable computing cores for the same graph region as the target computing core for the graph region (that is, the computing node).

[0094] S704: The task allocator merges the computing nodes with the same target computing core in each computing area into one graph area.

[0095] S705: The task allocator determines at least one target graph region according to the merged graph regions.

[0096] In some possible implementations, S704 to S705 may include: after determining the target computation core corresponding to each graph region, the task allocator may merge graph regions with the same target computation core into a larger graph region. When all graph regions cannot be further merged, the task allocator determines the graph regions corresponding to the computation graph as target graph regions.

[0097] Of course, S303 may also be implemented in other ways, which are not specifically limited in the embodiments of the present disclosure.

[0098] It should be noted that in some possible implementations, the task allocator can calculate an estimated execution time for a graph region in a mappable compute core by invoking a performance mathematical model. Optionally, the performance mathematical model is trained based on the historical execution times of scalar and / or tensor compute nodes in different types of compute cores. The execution time estimate obtained using the performance mathematical model is computationally simple and highly efficient, but has low accuracy.

[0099] In other possible implementations, the task dispatcher can also map the graph region to a mappable compute core's compiler, which generates a corresponding binary instruction sequence. This binary instruction sequence is then sent to the mappable compute core for execution, thereby determining the execution time of the graph region within the mappable compute core. While this method is more accurate than the aforementioned method of estimating execution time using a performance mathematical model, it is computationally complex and less efficient.

[0100] S304: The computing core compiling module generates a corresponding binary instruction code segment for each target graph region.

[0101] Among them, the binary code segment corresponding to the scalar calculation graph area can be executed in the first calculation core used for scalar calculation, and the binary code segment corresponding to the tensor calculation graph area can be executed in the second calculation core used for tensor calculation.

[0102] In some possible implementations, after determining at least one target graph region, the task dispatcher transmits the at least one target graph region and a mapping relationship between the target graph region and the compute core to a computing core compilation module. The computing core compilation module converts the at least one target graph region into a second intermediate representation based on the mapping relationship, and sends the second intermediate representation to a corresponding compiler to generate a binary instruction code segment corresponding to the target graph region.

[0103] S305 , the computing core compiling module generates a binary instruction sequence corresponding to the source language code based on the dependency relationship between the target graph regions and the binary instruction code segments.

[0104] It can be understood that the computing core compilation module mixes together the binary instruction code segments generated by each compiler, and adds the dependencies between each graph region and block to obtain a compiled binary instruction sequence.

[0105] At this point, the compilation process for heterogeneous systems is completed.

[0106] In some possible implementations, after the development process described in S301 to S305 is completed, the binary instruction sequence compiled in S305 can be mapped to a heterogeneous computing core for execution. Figure 1 As shown by the dashed line, the heterogeneous system 10 may further include a runtime system 13. During the actual operation, the programming program 11 sends the compiled binary instruction sequence to the heterogeneous computing core. During the sending process, the runtime system 13 may query whether there is a binary instruction code segment to be executed. When the runtime system 13 determines that the data required for the execution of the binary code segment to be executed is ready and the heterogeneous computing core corresponding to the binary instruction code segment is idle, the runtime system 13 may send the binary instruction code segment to the corresponding heterogeneous computing core for execution.

[0107] In the disclosed embodiments, when developing programs based on heterogeneous systems, since the source language code can describe the algorithm using both scalar and tensor programming languages, there is no need to program separately for different heterogeneous computing cores. Instead, scalar and tensor calculations can be written simultaneously during the programming process, greatly improving development efficiency. Furthermore, through computational graph partitioning, the same computational function can be mapped to different heterogeneous computing cores for execution, shortening the execution time of the function and improving the effective utilization of computing resources.

[0108] Based on the same inventive concept, an embodiment of the present disclosure provides a compiling device for compiling programs for a heterogeneous system.

[0109] Figure 9 This is a schematic diagram of the structure of a compilation device in an embodiment of the present disclosure, see Figure 9 As shown, the compilation device 900 may include: a programming front-end module 901, used to obtain a computational graph corresponding to a source language code, the source language code is written based on a programming model, and the programming model is algorithmically described by a scalar programming language and a tensor programming language; a computational graph processing module 902, used to divide the computational graph into at least one target graph area, at least one target graph area includes a scalar graph area containing scalar computing nodes and / or a tensor graph area containing tensor computing nodes; a computational core decoding module 903, used to generate a corresponding binary instruction code segment for each target graph area; and, based on the dependency relationship between the target graph areas and the binary instruction code segments, generate a binary instruction sequence corresponding to the source language code, the binary code segment corresponding to the scalar graph area can be executed in a first computing core used for scalar computing, and the binary code segment corresponding to the tensor graph area can be executed in a second computing core used for tensor computing.

[0110] In the above solution, the programming model includes a built-in function interface written in a tensor programming language that can be run in the second computing core, and the built-in function interface is used to be called to perform tensor calculations on tensor data.

[0111] In the above scheme, the programming front-end module 901 is used to convert the source language code into a first intermediate representation, which is used to represent scalar data and / or tensor data; and generate a computational graph based on the intermediate representation, wherein the computational nodes in the computational graph represent functions corresponding to the source language code, and the directed edges in the computational graph represent dependencies between functions.

[0112] In the above scheme, the programming front-end module 901 is also used to optimize the calculation graph; wherein the optimization processing includes at least one of the following processing: deleting redundant computing nodes, computing node fusion, equivalent conversion of complex calculations to simple calculations, and loop block merging.

[0113] In the above scheme, the computational graph processing module 902 is used to divide the computational nodes in the computational graph that have dependencies and the same computational type into a computational area; count the mappable computational cores of each computational node in each computational area; divide the computational nodes in each computational area that have dependencies and the same mappable computational core into a graph area; and determine at least one target graph area based on the graph area in each computational area.

[0114] In the above solution, the calculation area is a scalar calculation area; the calculation graph processing module 902 is used to determine at least one graph area corresponding to the calculation graph as at least one target graph area.

[0115] In the above scheme, the calculation area is a tensor calculation area; the calculation graph processing module 902 is used to create a graph area set for the i-th graph area in the graph area; traverse the graph area set, and divide the computing nodes in the i-th graph area that have dependencies and can be mapped to the same computing core into a candidate graph area, where i is a positive integer; determine the first benefit value of the i-th graph area based on the execution time of the candidate graph area of ​​the i-th graph area on the corresponding computing core; determine the second benefit value that is greater than the preset benefit value and has the largest value from the first benefit value; update the candidate graph area corresponding to the second benefit value to the graph area set, and return to the step of traversing the graph area set until the first benefit values ​​are all less than or equal to the preset benefit value; determine the candidate graph area corresponding to the first benefit value as the target graph area.

[0116] In the above scheme, the computational graph processing module 902 is also used to divide the computational nodes of the same type that have dependencies in the computational graph into a computational area; calculate the execution time of each computational node in each computational area in the mappable computational core, and determine the mappable computational core with the shortest execution time as the target computational core of the computational node; merge the computational nodes with the same target computational core in each computational area into a graph area; and determine at least one target graph area based on the merged graph area.

[0117] In the above scheme, the computing core decoding module 903 is used to determine the mapping relationship between each target graph area in at least one target graph area and the computing core; according to the mapping relationship, each target graph area is converted into a second intermediate representation; based on the second intermediate representation, a binary instruction code segment is generated.

[0118] Based on the same inventive concept, an embodiment of the present disclosure provides a compilation device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: implement the compilation method described in one or more of the above embodiments when executing the executable instructions.

[0119] Based on the same inventive concept, an embodiment of the present disclosure provides a computer-readable storage medium, which stores an executable program. When the executable program is executed by a processor, it implements the compilation method described in one or more of the above embodiments.

[0120] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0121] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A compilation method based on a heterogeneous system, wherein the heterogeneous system includes multiple computing cores with different architectures, characterized in that: The method comprises: Obtaining a computational graph corresponding to a source language code, wherein the source language code is written based on a programming model, and the programming model is algorithmically described using a scalar programming language and a tensor programming language; Divide the computing nodes in the computing graph that have dependencies and the same computing type into a computing area; Count the mappable computing cores of each computing node in each computing area; Divide computing nodes in each computing region that have dependencies and can map the same computing core into a graph region; Determining at least one target graph region according to the graph region in each computation region, wherein the at least one target graph region includes a scalar graph region including scalar computation nodes and / or a tensor graph region including tensor computation nodes; Generate a corresponding binary instruction code segment for each target graph region, wherein the binary code segment corresponding to the scalar graph region can be executed in a first computing core for scalar calculations, and the binary code segment corresponding to the tensor graph region can be executed in a second computing core for tensor calculations; generating a binary instruction sequence corresponding to the source language code based on the dependency relationship between the target graph regions and the binary instruction code segments; Wherein, the calculation area is a scalar calculation area; and determining at least one target graph area according to the graph area in each calculation area includes: determining the graph area as the at least one target graph area; The calculation area is a tensor calculation area; determining at least one target graph area based on the graph areas in each calculation area includes: creating a graph area set for the i-th graph area in the graph area; traversing the graph area set, dividing the computing nodes in the i-th graph area that have a dependency relationship and can be mapped to the same computing core into a candidate graph area, where i is a positive integer; determining the first benefit value of the i-th graph area based on the execution time of the candidate graph area of ​​the i-th graph area on the corresponding computing core; determining a second benefit value that is greater than a preset benefit value and has the largest value from the first benefit value; updating the candidate graph area corresponding to the second benefit value to the graph area set, and returning to the step of traversing the graph area set until the first benefit values ​​are all less than or equal to the preset benefit value; determining the candidate graph area corresponding to the first benefit value as the target graph area.

2. The method according to claim 1, characterized in that The programming model includes a built-in function interface written in the tensor programming language and capable of running in the second computing core, and the built-in function interface is used to be called to perform tensor calculations on tensor data.

3. The method according to claim 1, characterized in that Obtaining a computational graph corresponding to the source language code includes: Converting the source language code into a first intermediate representation, wherein the first intermediate representation is used to represent scalar data and / or tensor data; The computation graph is generated based on the first intermediate representation, wherein computation nodes in the computation graph represent functions corresponding to the source language code, and directed edges in the computation graph represent dependency relationships between the functions.

4. The method according to claim 3, characterized in that After generating the computation graph based on the first intermediate representation, the method further includes: Optimizing the computation graph; The optimization process includes at least one of the following processes: deleting redundant computing nodes, merging computing nodes, equivalent conversion of complex computing to simple computing, and merging loop blocks.

5. The method according to claim 1, wherein Generating a corresponding binary instruction code segment for each target image region includes: Determining a mapping relationship between each target graph area and a computing core in the at least one target graph area; Converting each target image region into a second intermediate representation according to the mapping relationship; Based on the second intermediate representation, the binary instruction code segment is generated.

6. The method according to any one of claims 1 to 5, characterized in that The first computing core includes at least one of the following computing cores: CPU, DSP; the second computing core includes at least one of the following computing cores: CPU, TPU, GPU, DSP, ASIC, FPGA.

7. A compilation device for compiling programs for a heterogeneous system, wherein the heterogeneous system includes multiple computing cores with different architectures, characterized in that: The compiling device comprises: A programming front-end module is used to obtain a computational graph corresponding to a source language code, wherein the source language code is written based on a programming model, and the programming model is algorithmically described using a scalar programming language and a tensor programming language; A computational graph processing module is configured to divide computational nodes in the computational graph that have a dependency relationship and the same computational type into a computational region; count the mappable computational cores of each computational node in each computational region; divide the computational nodes in each computational region that have a dependency relationship and the same mappable computational core into a graph region; and determine at least one target graph region based on the graph region in each computational region, the at least one target graph region including a scalar graph region containing scalar computational nodes and / or a tensor graph region containing tensor computational nodes; a computing core decoding module, configured to generate a corresponding binary instruction code segment for each target graph region; and, based on the dependency relationship between the target graph regions and the binary instruction code segments, generate a binary instruction sequence corresponding to the source language code, wherein the binary code segment corresponding to the scalar graph region can be executed in a first computing core for scalar calculations, and the binary code segment corresponding to the tensor graph region can be executed in a second computing core for tensor calculations; The calculation area is a scalar calculation area; the calculation graph processing module is further configured to determine at least one graph area corresponding to the calculation graph as the at least one target graph area; The calculation area is a tensor calculation area; the calculation graph processing module is also used to create a graph area set for the i-th graph area in the graph area; traverse the graph area set, and divide the computing nodes in the i-th graph area that have a dependency relationship and can be mapped to the same computing core into a candidate graph area, where i is a positive integer; determine the first benefit value of the i-th graph area based on the execution time of the candidate graph area of ​​the i-th graph area on the corresponding computing core; determine a second benefit value that is greater than a preset benefit value and has the largest value from the first benefit value; update the candidate graph area corresponding to the second benefit value to the graph area set, and return to the step of traversing the graph area set until the first benefit values ​​are all less than or equal to the preset benefit value; determine the candidate graph area corresponding to the first benefit value as the target graph area.

8. The device according to claim 7, characterized in that The programming model includes a built-in function interface written in the tensor programming language and capable of running in the second computing core, and the built-in function interface is used to be called to perform tensor calculations on tensor data.

9. The device according to claim 7, characterized in that The programming front-end module is used to convert the source language code into a first intermediate representation, which is used to represent scalar data and / or tensor data; generate the computational graph based on the first intermediate representation, wherein the computational nodes in the computational graph represent functions corresponding to the source language code, and the directed edges in the computational graph represent dependencies between the functions.

10. The device according to claim 9, characterized in that The programming front-end module is also used to optimize the computational graph after generating the computational graph based on the first intermediate representation; wherein the optimization processing includes at least one of the following processing: deleting redundant computing nodes, computing node fusion, equivalent conversion of complex calculations to simple calculations, and loop block merging.

11. The device according to claim 7, characterized in that The computing core decoding module is configured to determine a mapping relationship between each target graph region in the at least one target graph region and the computing core; and convert each target graph region into a second intermediate representation according to the mapping relationship; Based on the second intermediate representation, the binary instruction code segment is generated.

12. A compilation device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 6 when executing the executable instructions.

13. A computer-readable storage medium, characterized in that The readable storage medium stores an executable program, wherein the executable program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images