Dataflow graph programming environment for heterogeneous processing systems

Abstract

The examples herein describe techniques to generate a dataflow graph using source code to define kernels and communication links between those kernels. In one embodiment, the dataflow graph is formed by using nodes (e.g., kernels) communicably coupled by edges (e.g., communication links between kernels). A compiler converts the source code into a bitstream and / or binary code such that a heterogeneous processing system of a SoC is configured to execute the graph. The compiler uses the graph expressed in the source code to determine where to allocate the kernels in the heterogeneous processing system. Furthermore, the compiler can select specific communication techniques to establish the communication links between the kernels and whether synchronization should be used in the communication links. Thus, a programmer can express a dataflow graph at a high level (using source code) without needing to understand how to implement the computation graph by using the heterogeneous hardware in the SoC.

Description

TECHNICAL FIELD

[0001] Examples of the present disclosure generally relate to using object-oriented programming code to generate dataflow graphs in a system that includes a mix of programmable and software-configurable hardened hardware elements. BACKGROUND

[0002] A system on a chip (SoC) can include a mix of programmable logic (e.g., programmable fabric) and software-configurable hardened logic (e.g., processing cores or engines). Typically, a user must have detailed knowledge of the programmable and software-configurable hardened logic (and how they communicate) in order to write a program that can be compiled into a bitstream to configure the programmable and binary code to configure the software-configurable hardened logic to perform one user function. However, writing a program for a SoC with a mix of programmable and hardened logic using a hardware description language (HDL) or Open Computing Language (OpenCL) is both cumbersome and difficult to parallelize. Data parallelism and thread parallelism are also used to express computations on processor arrays, but these techniques do not naturally extend to programmable logic where heterogeneous computations with different interfaces need to be expressed. SUMMARY

[0003] This application describes techniques to implement dataflow graphs on a heterogeneous processing system. One example is a method that includes receiving graph source code that defines a plurality of kernels and a plurality of communication links, where each of the plurality of communication links couples a respective pair of the plurality of kernels to form a dataflow graph. The method also includes compiling the graph source code to implement the dataflow graph on a system in the heterogeneous processing system. Compiling the graph source code includes assigning the plurality of kernels to programmable logic and data processing engine (DPE) arrays in the heterogeneous processing system, assigning a communication type for the plurality of communication links, and selecting a synchronization technique to transfer data between the plurality of kernels using the plurality of communication links.

[0004] In some embodiments, assigning the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicably coupled by a first communication link of the plurality of communication links defined by the graph source code, assigning the first kernel to a first data processing engine (DPE) in the heterogeneous processing system, and assigning the second kernel to a second DPE in the heterogeneous processing system that is directly adjacent to the first DPE.

[0005] In some embodiments, the first DPE and the second DPE both have a direct connection to a shared memory module, and the method includes allocating a double buffer in the shared memory module for transferring data between the first kernel and the second kernel.

[0006] In some embodiments, allocating the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicatively coupled through a first one of the plurality of communication links defined by the graphical source code; allocating the first kernel to a first DPE in the heterogeneous processing system; allocating the second kernel to programmable logic in the heterogeneous processing system; and configuring the second kernel to perform direct memory access (DMA) using an interconnect to transfer data to the first kernel, wherein the interconnect interconnects an array of DPEs including the first DPE and connects to the programmable logic.

[0007] In some embodiments, allocating the plurality of kernels to the heterogeneous processing system includes identifying that a first kernel and a second kernel are communicatively coupled through a first one of the plurality of communication links defined by the graphical source code; clustering the first and second kernels to a first core in response to determining that the first kernel and the second kernel have a combined cycle count that is less than or equal to a cycle budget of the first core in an array of data processing engines in the heterogeneous processing system; and allocating a buffer in a memory module for transferring data between the first kernel and the second kernel, wherein the memory module has a direct connection to the first core.

[0008] In some embodiments, allocating a communication type to the plurality of communication links includes selecting whether to use one of streaming and windowing for transferring data for each of the plurality of communication links based on a definition of the plurality of communication links in the graphical source code.

[0009] In some embodiments, windowing includes dividing received data into separate windows having a pre-defined or parameterized block size, and each of the plurality of kernels configured to receive the separate windows waits until one window is received at each invocation before processing the received window. Further, for at least one of the communication links for which windowing is performed, a separate window has data at a beginning that overlaps an end of a previously transmitted window, such that a receiving kernel of the plurality of kernels that receives the separate window maintains its state.

[0010] In some embodiments, selecting a synchronization technique includes identifying a double buffer allocated to a first one of the plurality of communication links, and configuring a lock protocol to enable a first kernel and a second kernel corresponding to the first one of the plurality of communication links to access the double buffer in parallel.

[0011] In some embodiments, the method includes transmitting a bitstream and binary code based on the compiled graphical source code, the graphical source code configuring the heterogeneous processing system to execute the dataflow graph, and controlling execution of the dataflow graph in the heterogeneous processing system by using a control program.

[0012] In some embodiments, the heterogeneous processing system includes a first chip and a second chip, wherein the plurality of kernels is allocated to the first chip, the graph source code defines a second plurality of kernels, and compiling the graph source code includes allocating the second plurality of kernels to the second chip, wherein the second plurality of kernels allocated to the second chip are configured to communicate with the plurality of kernels allocated to the first chip.

[0013] In some embodiments, the graph source code is independent of a hardware design of a SoC forming the heterogeneous processing system, and can be implemented by the compiler onto a plurality of different types of SoCs each having a different hardware design.

[0014] In some embodiments, the heterogeneous processing system includes programmable logic and a DPE array, wherein at least one kernel of the plurality of kernels is allocated to the programmable logic, and at least one kernel of the plurality of kernels is allocated to one DPE of the DPE array.

[0015] In some embodiments, the method includes packaging a sub-graph into the dataflow graph, wherein the sub-graph is defined by a graph class separate from the graph source code, and generating a constraint graph that adds constraints to the dataflow graph and the sub-graph, wherein the constraint graph serves as a wrapper for the dataflow graph.

[0016] In some embodiments, each kernel of the plurality of kernels includes at least one port to enable each kernel of the plurality of kernels to communicate with another kernel in the dataflow graph, and in the dataflow graph, each communication link of the plurality of communication links couples a first port on a first kernel to a second port on a second kernel.

[0017] One example described herein is a host including a processor, a graph source code defining a plurality of kernels and a plurality of communication links, wherein each communication link of the plurality of communication links couples a respective pair of kernels of the plurality of kernels to form a dataflow graph, and a compiler configured to compile the graph source code to implement the dataflow graph in a heterogeneous processing system. Compiling the graph source code includes allocating the plurality of kernels to programmable logic and a DPE array in the heterogeneous processing system; allocating a communication type for the plurality of communication links; and selecting a synchronization technique for transferring data between the plurality of kernels using the plurality of communication links. BRIEF DESCRIPTION OF DRAWINGS

[0019] So that the manner in which the above-recited features can be understood in detail, a brief description of the examples illustrated herein follows, simply by way of examples, and with reference to the accompanying drawings, in which

[0020] Figure 1is a block diagram of a SoC including an array of data processing engines according to an example.

[0021] Figure 2 is a block diagram of a data processing engine in an array of data processing engines according to an example.

[0022] Figure 3A and 3B shows a memory module shared by multiple DPEs in a DPE array according to an example.

[0023] Figure 4 is a kernel source code for defining a kernel in a dataflow graph according to an example. Figure 1 is a block diagram of a computing system for implementing a dataflow graph on a SoC as shown.

[0024] Figure 5 is a flowchart for compiling source code to implement a dataflow graph on a SoC having programmable and non-programmable logic according to an example.

[0025] Figure 6 is a graph source code for defining a dataflow graph according to an example.

[0026] Figure 7 is a dataflow graph defined by source code in Figure 6 according to an example.

[0027] Figure 8 is a kernel source code for defining a kernel in a dataflow graph according to an example.

[0028] Figure 9 is an abstract view of implementing a dataflow graph in Figure 7 according to an example.

[0029] Figure 10 is a hardware view of implementing a dataflow graph in Figure 7 according to an example.

[0030] Figure 11 shows an overlap window used when transferring data between kernels according to an example.

[0031] Figure 12 is a control source code for defining a control program for a dataflow graph according to an example.

[0032] Figure 13 is a flowchart for compiling source code to implement a dataflow graph by using constraints according to an example.

[0033] Figure 14 is a DPE array having graph objects implemented using user-defined constraints according to an example.

[0034] Figure 15 is an inheritable abstract interface 1505 according to an example.

[0035] Figure 16 is a dataflow graph with multiple subgraphs according to an example.

[0036] Figure 17 is a constrained dataflow graph according to an example.

[0037] Figure 18 is a constraint processing flow for merging constraint conditions from multiple sources according to an example.

[0038] Figure 19 is a block diagram of a computing system for implementing a dataflow graph on a SoC according to an example.

[0039] Figure 20A and 20B shows a control application program interface for controlling execution of a dataflow graph on a SoC according to an example.

[0040] Figure 21 shows logically dividing a data processing engine array into different regions according to an example.

[0041] Figure 22A and 22B shows dynamically changing execution of a dataflow graph according to an example.

[0042] Figure 23A and 23B shows triggered and asynchronous parameters according to an example.

[0043] For ease of understanding, the same reference numbers will be used in different drawings to designate the same elements when understood from the context of such use. It is contemplated that elements from one example can be advantageously combined in a second example.

[0044] DETAILED DESCRIPTION

[0045] Various features are described below in connection with the following figures. It is noted that the figures can be drawn to scale and that like reference numbers in different figures represent similar elements. It is further noted that the figures are merely intended to facilitate the description of the features and are not intended to limit the scope of the claims. Additionally, the examples shown need not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular example is not necessarily limited to that example and can be practiced in any other examples even if not so described or even if not so explicitly described.

[0046] The examples herein describe techniques for using source code to generate a dataflow graph to define kernels and communication links between the kernels. In an embodiment, the graph is formed by using nodes (e.g., kernels) communicatively coupled by edges (e.g., communication links between kernels). A compiler converts the source code into bitstream and binary code that configures programmable logic and software-configurable hardened logic in a heterogeneous processing system of a SoC to execute the graph. The compiler does not require the programmer to have detailed knowledge of the programmable and software-configurable hardened hardware in the heterogeneous processing system, but rather can use the graph represented in the source code to determine which kernels to assign to programmable logic blocks and which kernels to assign to hardened logic blocks. In addition, the compiler can use parameters provided in the graph source code to select specific communication techniques to establish communication links between the kernels (e.g., shared memory, windowing, direct memory access (DMA), etc.). Furthermore, the compiler can automatically determine whether synchronization should be used in the communication links and set that synchronization without programmer input - i.e., the programmer does not need to provide detailed information of the synchronization in the graph source code. Thus, the programmer can express the dataflow graph at a high level (using source code) without needing to know how to implement the dataflow graph using programmable and hardened hardware in the SoC. As a result, the graph source code is independent of the hardware design of a particular SoC and can be implemented (using the compiler) to a variety of different types of SoCs, each having a different hardware design.

[0047] Figure 1 A block diagram of a SoC 100 including a data processing engine (DPE) array 105 according to an example. The DPE array 105 includes a plurality of DPEs 110, which can be arranged in a grid, cluster, or checkerboard pattern in the SoC 100. Although the DPEs 110 are shown arranged in a 2D array having rows and columns, embodiments are not limited to such an arrangement. In addition, the array 105 can be any size and have any number of rows and columns formed of DPEs 110. Figure 1 Although the DPEs 110 are shown arranged in a 2D array having rows and columns, embodiments are not limited to such an arrangement. In addition, the array 105 can be any size and have any number of rows and columns formed of DPEs 110.

[0048] In an embodiment, the DPEs 110 are identical. That is, each DPE 110 (also referred to as a tile or block) can have the same hardware components or circuitry. In addition, embodiments herein are not limited to DPEs 110. Rather, the SoC 100 can include an array of any kind of processing element, e.g., the DPEs 110 can be digital signal processing engines, cryptographic engines, forward error correction (FEC) engines, or other specialized hardware for performing one or more specialized tasks.

[0049] In an embodiment, the DPEs 110 are identical. That is, each DPE 110 (also referred to as a tile or block) can have the same hardware components or circuitry. In addition, embodiments herein are not limited to DPEs 110. Rather, the SoC 100 can include an array of any kind of processing element, e.g., the DPEs 110 can be digital signal processing engines, cryptographic engines, forward error correction (FEC) engines, or other specialized hardware for performing one or more specialized tasks. Figure 1In particular embodiments, array 105 includes DPEs 110 of all the same type (e.g., a homogeneous array). However, in another embodiment, array 105 can include different types of engines. For example, array 105 can include digital signal processing engines, encryption engines, graphics processing engines, etc. As described in more detail below, regardless of whether array 105 is homogeneous or heterogeneous, DPEs 110 can include direct connections between DPEs 110 that allow DPEs 110 to directly transfer data.

[0050] In an embodiment, DPEs 110 are formed from software-configurable hardened logic - i.e., hardened. One advantage of doing so is that DPEs 110 can take up less space in SoC 100 relative to using programmable logic to form the hardware elements in DPEs 110. That is, using hardened logic circuits to form hardware elements in DPEs 110 such as program memory, instruction fetch / decode units, fixed-point vector units, floating-point vector units, arithmetic logic units (ALUs), multiply-accumulators (MACs), etc. can significantly reduce the footprint of array 105 in SoC 100. Although DPEs 110 can be hardened, this does not mean that DPEs 110 are unprogrammable. That is, DPEs 110 can be configured to perform different functions or tasks upon power-up or reboot of SoC 100.

[0051] DPE array 105 also includes SoC interface blocks 115 (also referred to as pads) that serve as communication interfaces between DPEs 110 and other hardware elements in SoC 100. In the present example, SoC 100 includes a network-on-chip (NoC) 120 that is communicatively coupled to SoC interface blocks 115. Although not shown, NoC 120 can extend throughout SoC 100 to allow various elements in SoC 100 to communicate with each other. For example, in one physical implementation, DPE array 105 can be arranged in the upper right portion of an integrated circuit that forms SoC 100. However, using NoC 120, array 105 can still communicate with, for example, programmable logic (PL) 125, processor subsystem (PS) 130, or input / output (I / O) 135, which can be arranged in different locations throughout SoC 100.

[0052] In addition to providing an interface between DPEs 110 and NoC 120, SoC interface blocks 115 can also provide a direct connection to the communication fabric in PL 125. In the present example, PL 125 and DPEs 110 form a heterogeneous processing system, as some of the kernels in the dataflow graph can be assigned for execution by DPEs 110 while others are assigned to PL 125. Although Figure 1A heterogeneous processing system is shown placed into an SoC, but in other examples the heterogeneous processing system can include multiple devices or chips. For example, the heterogeneous processing system can include two FPGAs of the same type or different types or other specialized accelerator chips. Further, the heterogeneous processing system can include two SoCs communicably coupled.

[0053] This can be difficult for a programmer to manage as communication between kernels arranged in heterogeneous or different processing cores can include using Figure 1 various communication interfaces shown in FIG. 1, such as the NoC 120, the SoC interface block 115, and communication links between the DPEs 110 in the array 105 (as shown in FIG. 2). Figure 2

[0054] In an embodiment, the SoC interface block 115 includes a separate hardware component for communicably coupling the DPEs 110 to the NoC 120 and to the PL 125 disposed proximate to the array 105 in the SoC 100. In an embodiment, the SoC interface block 115 can stream data directly to the fabric for the PL 125. For example, the PL 125 can include an FPGA fabric, and the SoC interface block 115 can stream data into and receive data from the fabric without using the NoC 120. That is, circuit switching and packet switching described herein can be used to communicably couple the DPEs 110 to the SoC interface block 115 and other hardware blocks in the SoC 100. In another example, the SoC interface block 115 can be implemented in a different die than the DPEs 110. In yet another example, the DPE array 105 and at least one sub-system can be implemented in the same die, while other sub-systems and / or other DPE arrays are implemented in other dies. Further, the flow interconnect and routing described herein with respect to the DPEs 110 in the DPE array 105 can also apply to routing data through the SoC interface block 115.

[0055] Although Figure 1 One PL 125 block is shown, but the SoC 100 can include multiple PL 125 blocks (also referred to as configuration logic blocks) that can be disposed in different locations in the SoC 100. For example, the SoC 100 can include hardware elements that form a field programmable gate array (FPGA). However, in other embodiments, the SoC 100 can not include any PL 125 - for example, the SoC 100 is an ASIC.

[0056] Figure 2 is in accordance with an example. Figure 1 ​The diagram shows a block diagram of DPE 110 in DPE array 105. DPE 110 includes interconnect 205, core 210, and storage module 230. Interconnect 205 allows data to be transferred from core 210 and storage module 230 to different cores in array 105. That is, the interconnect 205 in each DPE 110 can be interconnected, so that data can be transferred north and south (e.g., up and down) and east and west (e.g., right and left) in the array of DPE 110.

[0057] Come back for reference Figure 1 In one embodiment, the upper row of DPEs 110 in array 105 communicates with the SoC interface block 115 via interconnects 205 in the lower row of DPEs 110. For example, to transfer data to the SoC interface block 115, cores 210 in the upper row of DPEs 110 transfer data to their interconnects 205, which are communicatively coupled to interconnects 205 in the lower row of DPEs 110. The interconnects 205 in the lower row are connected to the SoC interface block 115. This process can be reversed when data intended for the upper row of DPEs 110 is first transferred from the SoC interface block 115 to the interconnects 205 in the lower row and then to the upper row of interconnects 205 that are the target DPEs 110. In this way, the upper row of DPEs 110 can rely on the interconnects 205 in the lower row of DPEs 110 to send data to and receive data from the SoC interface block 115.

[0058] In one embodiment, interconnect 205 includes a configurable switching network that allows a user to determine how data is routed through interconnect 205. In one embodiment, unlike a packet-routing network, interconnect 205 can form flow point-to-point connections. That is, flow connections and flow interconnects in interconnect 205 (… Figure 2 (Not shown) Routes can be formed from core 210 and storage module 230 to adjacent DPE 110 or SoC interface block 115. Once configured, core 210 and storage module 230 can send and receive streaming data along these routes. In one embodiment, interconnect 205 is configured using the Advanced Extensible Interface (AXI) 4 streaming protocol.

[0059] In addition to forming the flow network, interconnect 205 may include separate networks for programming or configuring hardware elements in DPE 110. Although not shown, interconnect 205 may include memory-mapped interconnects that include different connections and switching elements for setting values ​​of configuration registers in DPE 110 to change or set the functionality of the flow network, core 210, and storage module 230.

[0060] In an embodiment, the stream interconnect (or network) in interconnect 205 supports two different modes of operation, referred to herein as circuit switched and packet switched. In an embodiment, both of these modes are part of or compatible with the same stream protocol (e.g., AXI stream protocol). Circuit switched relies on a point-to-point communication path that is reserved between a source DPE 110 to one or more target DPEs 110. In an embodiment, the point-to-point communication path used when performing circuit switched in interconnect 205 is not shared with other streams (whether circuit switched or packet switched). However, when transferring stream data between two or more DPEs 110 by using packet switched, the same physical wire can be shared with other logical streams.

[0061] Core 210 can include hardware elements for processing digital signals. For example, core 210 can be used to process signals related to wireless communications, radar, vector operations, machine learning applications, etc. Thus, core 210 can include program memory, instruction fetch / decode units, fixed point vector units, floating point vector units, arithmetic logic units (ALUs), multiply-accumulators (MACs), etc. However, as mentioned above, the present disclosure is not limited to DPEs 110. The hardware elements in core 210 can vary with the type of engine. That is, the cores in a digital signal processing engine, a cryptographic engine, or an FEC can be different.

[0062] Storage module 230 includes a direct memory access (DMA) engine 215, memory banks 220, and a hardware synchronization circuit (HSC) 225, or other type of hardware synchronization block. In an embodiment, DMA engine 215 enables data to be received and transmitted by interconnect 205 to interconnect 205. That is, DMA engine 215 can be used to perform DMA reads and writes to memory banks 220 by using data received from a SoC interface block or other DPE 110 in the array via interconnect 205.

[0063] Memory banks 220 can include any number of physical storage elements (e.g., SRAM). For example, storage module 230 can include 4, 8, 16, 32, etc. different memory banks 220. In the present embodiment, core 210 has a direct connection 235 to memory banks 220. In other words, core 210 can write data to or read data from memory banks 220 without using interconnect 205. That is, direct connection 235 can be separate from interconnect 205. In an embodiment, one or more wires in direct connection 235 communicably couple core 210 to a memory interface in storage module 230 to which core 210 is connected, which in turn is coupled to memory banks 220.

[0064] In an embodiment, the storage module 230 also has direct connections 240 to the cores in the neighboring DPEs 110. In other words, neighboring DPEs in the array can read data from or write data to the memory banks 220 using the direct neighboring connections 240 without having to rely on their interconnects or Figure 2 The HSC 225 can be used to manage or protect access to the memory banks 220. In an embodiment, the HSC 225 provides a lock to an allocated portion of the memory banks 220 (referred to as a “buffer”) before a core 210 or a core in a neighboring DPE can read data from or write data to the memory banks 220. That is, when the core 210 wants to write data, the HSC 225 provides a lock to the core 210, which allocates a portion of one memory bank 220 (or multiple memory banks 220) to the core 210. Once the write is complete, the HSC 225 can release the lock to allow a core in a neighboring DPE to read the data.

[0065] Because the cores 210 and the cores in the neighboring DPEs 110 can directly access the storage module 230, the memory banks 220 can be considered shared memory between the DPEs 110. That is, a neighboring DPE can directly access the memory banks 220 in the same way that the cores 210 in the same DPE 110 access the memory banks 220. Thus, if the core 210 wants to transfer data to a core in a neighboring DPE, the core 210 can write the data into the memory banks 220. The neighboring DPE can then retrieve the data from the memory banks 220 and begin processing the data. In this way, the cores in the neighboring DPEs 110 can transfer data using the HSC 225 while avoiding the additional latency introduced when using the interconnects 205. Conversely, if the core 210 wants to transfer data to a non-neighboring DPE in the array (i.e., a DPE that does not have a direct connection 240 to the storage module 230), the core 210 routes the data to the target DPE’s storage module using the interconnects 205, which can take longer to complete because of the added latency using the interconnects 205 and because the data is copied into the target DPE’s storage module rather than being read from a shared storage module.

[0066] In addition to shared memory module 230, core 210 can be directly connected to cores 210 in adjacent DPEs 110 through the use of core-to-core communication links (not shown). That is, instead of using shared memory module 230 or interconnect 205, core 210 can directly transfer data to another core in the array without storing the data in memory module 230 or using interconnect 205 (which can have a buffer or other queue). For example, using a core-to-core communication link for communication can use less latency (or have high bandwidth) compared to using interconnect 205 or shared memory (which requires one core to write the data and then another core to read the data), which can provide a more cost effective communication. In an embodiment, a core-to-core communication link can transfer data between two cores 210 in one clock cycle. In an embodiment, data is transferred between cores on the link without being stored in any memory element external to core 210. In an embodiment, core 210 can send a data word or vector to an adjacent core every clock cycle using the link, although this is not required.

[0067] In an embodiment, the communication link is a streaming data link that allows core 210 to transfer a stream of data to an adjacent core. In addition, core 210 can include any number of communication links that can scale to different cores in the array. In this example, DPE 110 has various core-to-core communication links to cores in DPEs located to the left (west) and right (east) and above (north) or below (south) of core 210 in the array. However, in other embodiments, Figure 2 The illustrated cores 210 in DPE 110 can also have core-to-core communication links to cores disposed diagonally from core 210. In addition, if core 210 is disposed on the bottom periphery or edge of the array, the core can have core-to-core communication links to cores to the left, right, and bottom of core 210 only.

[0068] However, if the destination of the data generated by core 210 is an adjacent core or DPE, the use of shared memory in memory module 230 or core-to-core communication link can be useful. For example, if the destination of the data is a non-adjacent DPE (i.e., any DPE that does not have a direct adjacent connection 240 or core-to-core communication link to DPE 110), core 210 uses interconnect 205 in the DPE to route the data to the appropriate destination. As described above, interconnect 205 in DPE 110 can be configured at the time the SoC is booted to establish a point-to-point streaming connection to a non-adjacent DPE to which core 210 will transfer data during operation.

[0069] Figures 3A-3BA shared memory module 230A for a plurality of DPEs 110 in a DPE array according to an example is shown. As shown, the memory module 230A is directly connected to four cores - namely, cores 210A-D. The memory module 230A is in the same DPE as core 210A (i.e., DPE 110A). Thus, the direct connections 235 are intra-engine connections. However, the memory module 230A is in a different DPE than cores 210B-D. Thus, the directly adjacent connections 240A-C are inter-engine connections because these connections 240 cross the interface between DPEs 110 in the array. The interconnections in each DPE 110 are omitted for clarity.

[0070] In Figure 3A the memory module 230A in DPE 110A is disposed to the right of core 210A. The same is true for DPE 110D, which is to the right (i.e., east) of DPE 110A. Thus, core 210D in DPE 110D is directly adjacent to memory module 230A, which makes it easier to establish a directly adjacent connection 240B between memory module 230A and core 210D than if memory module 230D were disposed to the left of core 210D - i.e., if memory module 230D were disposed between memory module 230A and core 210D.

[0071] Unlike DPEs 110A and 110D, in DPEs 110B and 110C, cores 210B and 210C are disposed to the right of memory modules 230B and 230C. Thus, cores 210B and 210C are directly disposed above and below memory module 230A (i.e., cores 210B and 210C are to the north and south of memory module 230A). Doing so makes it easier to establish directly adjacent connections 240A and 240C between shared memory module 230A and cores 210B, 210C than if cores 210B and 210C were disposed to the left of memory modules 230B and 230C. Using Figure 3A With the arrangement shown in Figure 3A While the shared memory module 230A is shown as being shared among four cores 210, in other embodiments, the memory module 230A can be shared by more or fewer cores. For example, the memory module 230A can also be directly connected to adjacent DPEs that are disposed diagonally relative to DPE 110A.

[0072] Figure 3AThe arrangement of DPE 110 shown is merely an example of a suitable arrangement of DPE 110 to provide a direct connection from adjacent core 210 to storage module 230A. Figure 3B In this arrangement, DPE 110s in different rows are staggered. That is, instead of aligning DPE 110s in the same column, they are offset. In this arrangement, cores 210B and 210C are positioned to the left of storage modules 230B and 230C (unlike...). Figure 3A (As shown in the diagram) and by moving DPEs 110B and 110C to the right relative to DPE 110A, it remains directly above and below the shared storage module 230A. Therefore, direct connections 240A-C can be formed in the SoC so that storage module 230A can be shared by cores 210A-D.

[0073] In addition, although Figure 3A and 3B As not shown, storage modules 230B-D can also be shared storage modules. For example, storage module 230D can be directly connected to cores in a DPE located above, below, and to the right (i.e., north, south, and east) of DPE 110D. In this way, storage module 230D can be shared with cores in adjacent DPEs. However, storage modules 230 in DPEs located at the array edge or periphery may be shared by a smaller number of cores (or may not be shared at all).

[0074] Figure 4 It is based on the example used in Figure 1 The diagram shows a block diagram of a computing system 400 implementing a dataflow graph 440 on a SoC 100. System 400 includes a host 405 (e.g., a host computing system) which includes a processor 410 and memory 415. The processor 410 represents any number of processing elements, each of which may contain any number of processing cores. Memory 415 may include volatile and non-volatile memory elements. Furthermore, memory 415 may be located within the same device (e.g., a server) or may be distributed across computing system 400 (e.g., a cloud computing environment).

[0075] Memory 415 includes a heterogeneous programming environment 417 for generating graph source code 420, kernel source code 425, and control source code 430. Memory 415 also includes a compiler 435. Graph source code 420 can be written in various types of object-oriented programming languages ​​(e.g., C++, Python, Javascript, Swift, Go, LabVIEW, or Simulink). Typically, graph source code 420 defines kernels (e.g., nodes) connected via communication links (e.g., edges). The combination of kernels and communication links forms graph 440.

[0076] One advantage of providing a heterogeneous programming environment 417 for defining dataflow graphs 440 by using source code 420 is that different aspects of compiling dataflow graphs on a heterogeneous processing system can be directly expressed and controlled in the heterogeneous programming environment 417. A programmer can start with a parallel definition (e.g., a graph) and then the compiler 435 implements the definition in the hardware of the SoC 100. The graph 440 enables data to flow between nodes (e.g., kernels) in a continuous pipeline fashion. A node starts processing as soon as data is available at the node input, otherwise it stalls. In addition, the graph 440 provides the programmer with a great degree of freedom to map computations and dataflow to DPEs 110 and programmable logic 125 in the SoC 100.

[0077] While various types of dataflow graphs can be used, in one embodiment the semantics of the graph 440 established by the graph source code 420 are based on the general theory of Kahn Process Networks, which provides a computational model for deterministic parallel computation suitable for the heterogeneous architecture in the SoC 100 (including programmable blocks and hard blocks). In addition, the graph source code 420 tolerates communication delays between nodes in the graph 440, thus it naturally extends to graphs mapped to multiple super logic regions and multiple SoC devices (e.g., multiple FPGAs). For example, the graph source code 420 can include a first plurality of kernels assigned by the compiler to a first chip (e.g., SoC, FPGA, etc.) and a second plurality of kernels assigned by the compiler to a second chip. The first and second plurality of kernels can be part of the same dataflow graph, thus they can communicate with each other when executed on the first and second chips.

[0078] Another advantage of using the source code 420 to define dataflow graphs is that sequential programs fix the control flow and order of computation by comparison. With dataflow graphs, a predictable and reproducible response to input is obtained without race conditions. While there is a risk of deadlock, this risk can be solved or reduced by managing the storage allocated to each node or kernel.

[0079] The kernel source code 425 can be written in various types of object-oriented programming languages. The kernel source code 425 defines the properties of a particular kernel or node in the dataflow graph 440. In one embodiment, the kernel source code 425 defines the operations of each kernel within the graph source code 420.

[0080] The control source code 430 can be written in various types of object-oriented programming languages. In an embodiment, the control source code 430 defines a control program that, when executed, controls the execution of the graph 440 when implemented on the SoC 100. For example, the control source code 430 can control when the graph 440 executes, the number of iterations the graph 440 executes, and when the graph 440 stops executing. The control program generated from the control source code 430 can be executed on the host 405 (e.g., in a data center solution) or within the SoC 100 (e.g., the PS 130).

[0081] The compiler 435 is a software application that can compile the source code 420, 425, and 430. For example, using the graph source code 420 (as well as other libraries not shown in FIG. 4), the compiler 435 can generate a graph 440 that can be implemented on the SoC 100, which will be described in more detail below. In an embodiment, the graph 440 includes a bitstream 445 that configures programmable logic in the SoC 100 (e.g., the PL 125, the NoC 120, the SoC interface block 115, and the I / O 135) and a binary code 447 that configures software-configurable hardened logic in the SoC 100 (e.g., the DPEs 110 and the PS 130) that can include a number of target commands. The bitstream 445 and the binary code 447 can be transmitted to the SoC 100 over a memory bus to configure the SoC 100 to execute the graph 440. Figure 4

[0082] Figure 5 is a flowchart of a method 500 for compiling source code to implement a dataflow graph on a SoC having programmable logic and software-configurable hardened logic, according to an example. In block 501, a host provides a heterogeneous programming environment for defining a dataflow graph as object-oriented source code (e.g., C++, Python, Javascript, Swift, Go, LabView, or Simulink). That is, a programmer uses the heterogeneous programming environment (described in more detail in FIG. 5) to generate source code that defines a dataflow graph. In block 505, a compiler receives the source code that establishes a dataflow graph that defines kernels and communication links between the kernels. In an embodiment, the source code received by the compiler includes graph source code. Figure 6

[0083] For clarity, Figures 6-11 is discussed with the blocks described in method 500.

[0084] Figure 6 is graph source code 420 for defining a dataflow graph, according to an example. That is, Figure 6 ​​is one example of graph source code 420 generated in a heterogeneous programming environment that allows a programmer to define multiple kernels and communication links to establish a dataflow graph. The source code 420 using the namespace "namespace A" can reference one or more libraries available to define the dataflow graph in the source code 420. In an embodiment, the graph source code 420 can be considered to establish a data structure in a heterogeneous programming environment that the programmer builds by using the kernels 605 and communication links 620.

[0085] In this example, the graph source code 420 includes six kernels 605: a, b, c, d, e, and f. The kernels 605 are defined in the "radio" category. While the source code 420 is shown for performing a radio function, as described above, the techniques described herein can be used for a variety of different functions, such as radar, vector operations, machine learning applications, etc. Figure 6 The source code 420 is shown for performing a radio function, as described above, but the techniques described herein can be used for a variety of different functions, such as radar, vector operations, machine learning applications, etc.

[0086] The source code 420 includes wrappers 610A-F that define the function or operation performed by each kernel 605. The wrappers 610 create a mechanism to call a corresponding C++ function (e.g., polarclip, feedback, equalizer, fir tap 11, fir tap 7, and scale). That is, the wrappers 610 allow the programmer to define a kernel using an example function that can be part of another C++ library. In this example, the kernels 605 are function calls rather than single instructions. In an embodiment, a kernel 605 only executes when the kernel 605 receives data from all of its trigger inputs and executes in a non-blocking manner to produce output that can be transmitted to a downstream kernel 605. A kernel can also block during execution of the stream input if there is no stream data available at the time of access.

[0087] One advantage of using wrappers 610 to abstract kernels as function calls is that doing so means that the programmer can express kernels that will execute on DPEs or programmable logic in the same uniform framework. The programmer writes the kernels 605 differently, but the kernels 605 are packaged and can be expressed in the same framework the same way. The programmer does not need to worry about how to integrate kernels assigned to DPEs with kernels assigned to PL fabric. Here, the programmer selects or indicates the type of communication link 620 in the graph source code 420, and all synchronization between kernels 605 that use these types of communication links 620 is handled by the compiler.

[0088] Source code 420 also includes constraints 615, which include instructions that limit how the compiler maps objects defined in source code 420 (e.g., kernels 605 and communication links 620) to hardware in the SoC. In this example, constraints 615 instruct the compiler to allocate kernels a and f to fabric (e.g., programmable logic) in the SoC rather than to DPEs. Allocating kernels a and f to fabric rather than to DPEs can provide performance improvements for the reasons described below. Thus, while graphical source code 420 does not require the programmer to allocate kernels 605 to hardware in the SoC (and thus the programmer does not need to understand the underlying hardware architecture of the SoC), the namespace provided to the programmer allows it to use constraints 615 to instruct the compiler how to allocate one or all of kernels 605 if the programmer knows that doing so will improve performance.

[0089] Communication links 620 define how data is communicated between kernels 605. For example, communication link 620A instructs that streaming data is converted to windowed data having a length of 64 bytes. Further, each window is transmitted with an 8 byte overlap. However, for communication link 620B, windowed data having a length of 32 bytes is transmitted between kernel b and kernel c without any overlapping data. Details of windowed data (and overlapping windows) are described in more detail below.

[0090] Further, each communication link 620 defines which port on an upstream kernel is connected to which port on a downstream kernel. For example, in link 620A, output port a.out[0] of kernel a is coupled to input port b.in[0] of kernel b. Each kernel can have multiple input ports and multiple output ports. For example, in communication link 620D, first output port d.out[1] of kernel d is coupled to input port e.in[0]. Further, in communication link 620F, second output port d.out[0] of kernel d is coupled to input port f.in[0].

[0091] Similar to how graphical source code 420 abstracts kernels 605 so that they can be expressed in the same unified framework, source code 420 can abstract (or hide) synchronization on communication links 620 from the programmer. As described in more detail below, the compiler can select the best communication technique to transmit data between kernels 605 based on whether the kernels 605 are in fabric or in a DPE array, or whether the kernels 605 are adjacent in a DPE array.

[0092] In an embodiment, the ability to define kernels 605, wrappers 610, constraints 615, and communication links 620 in graphical source code 420 is provided by a tool (and supported by libraries in the namespace) of a heterogeneous programming environment that allows a programmer to generate object-oriented source code that implements a dataflow graph.

[0093] Figure 7 It is based on the example. Figure 6 The data flow graph 440 is defined in source code 420. That is, graph 440 is a graphical representation of the graph defined in source code 420. As shown, graph 440 includes six kernels af that are communicatively coupled using communication links 620A-E. Furthermore, graph 440 includes an input terminal 705 that transmits data to kernel a and an output terminal 710 that receives data from the output of kernel f. The data received at input terminal 705 can be provided by, for example, an application running on the host, a radio transceiver, a camera, or come from a file or database. Output terminal 710 can transmit the data processed by graph 440 to the host or add it to a file or database.

[0094] Figure 7 This is an abstract view of graph 440, where kernels (e.g., nodes) are coupled via link 620 at their respective input and output ports. That is, Figure 7 The diagram illustrates the data flow between kernels af using links 620A-F, but it does not show the hardware implementation executing the kernel or the specific type of communication link 620 being used—such as shared memory, NoC, DMA, etc. Nevertheless, programmers can... Figure 7 The abstract view shown depicts a design of graphic 440, from which the compiler can then implement the kernel AF and communication link 620 within the SoC's hardware.

[0095] Figure 8 This is based on kernel source code 425, used to define the kernel in a data flow graph, as shown in the example. In one embodiment, Figure 6 Wrapper 610 in the source code allows kernel-defined function parameters to be accessed as ports. Figure 8 In kernel source code 425, parameter 805 specifies a pointer to input data (i.e., *inputw) and a pointer to output data (*outputw). When two kernels are communicatively coupled via the link described above, the compiler can allocate data memory provided to the kernel (or functions called by the kernel) when the kernel is invoked. In one embodiment, the kernel manipulates the input data provided by parameter 805 using an application programming interface (API).

[0096] exist Figure 8 In kernel source code 425, there are window APIs for processing input data before output. For example, `window_readincr` is an API that uses the pointer `inputw` to read the next window. Once the operation is performed, this is typically shown here as performing mathematical operations using `sbuff`, and another API can be used to output the processed data—for example, `window_writeincr`.

[0097] In one embodiment, the programmer generates kernel source code for each kernel defined in the graph source code. However, if the graph source code has multiple instances of the same kernel, the multiple instances can be defined by using the same kernel source code.

[0098] Returning to the method 500, in block 510, the compiler compiles the source code (e.g., graph, kernel, and control source code). For ease of explanation, this compilation is divided into at least three sub-blocks. In block 515, the compiler allocates the kernels to the DPEs and programmable logic in the SoC. The compiler can use constraints provided by the programmer in the source code (e.g., constraints 615 in FIG. 6), but if no constraints are provided, the kernels in the graph source code can be allocated to the DPEs and programmable logic in the SoC. Figure 6

[0099] In one embodiment, the compiler evaluates the graph to determine how to allocate the kernels to the hardware in the SoC. For example, if two kernels are communicatively coupled to each other in the graph, the compiler can allocate the kernels to adjacent DPEs in the DPE array to take advantage of a faster communication protocol, such as shared memory between the DPEs. In addition, the compiler can determine the loop count and time fraction used by each kernel to determine whether multiple kernels can be allocated to the same DPE.

[0100] Figure 9 is an abstract view of the dataflow graph 440 in accordance with an example. Figure 7 is an abstract view of the dataflow graph 440 in accordance with an example. Figure 9 shows the kernels a-f and the communication links 620. In addition, Figure 9 shows the hardware in the SoC to which the kernels are allocated. As shown, the kernels a and f are disposed in the PL 125, kernels b and c are implemented in the DPE 110A, and kernels d and e are implemented in the DPE 110B.

[0101] In one embodiment, the compiler selects to place the kernels a and f in the PL 125 based on constraints provided in the graph source code. However, in another embodiment, the compiler can have identified these kernels as input / output kernels that can be more suitable to implement in programmable logic rather than in a DPE.

[0102] ​The compiler can have assigned kernels b and c to the same DPE 110A by using an estimated fraction of the loop count of each kernel or in response to constraints from the programmer. This is commonly referred to as clustering. For example, if kernel b uses only 40% of the loop count of DPE 110A and kernel c uses only 55% of the loop count, the compiler can place them on the same DPE 110A. In another example, the programmer can use constraints to instruct the compiler to place kernels b and c on the same DPE 110A. In this way, although the programmer describes the graph as a parallelized data structure, the programmer can use the estimated cycle count of the kernels to force certain kernels to be sequential - i.e., assigned to the same DPE. That is, because each DPE can only perform one task at a time (i.e., cannot be parallelized), placing two different kernels on the same DPE means that only one kernel can be executed at a time as opposed to the scenario where the kernels are assigned to their own DPE. However, this clustering will still satisfy the total loop count.

[0103] Returning to the method 500, in block 520, the compiler assigns the connections between the kernels to streaming or windowing. In one embodiment, these connections are controlled by the communication links defined in the graph source code. That is, the programmer can indicate how data should be passed between each pair of kernels. In another example, the compiler assigns a DMA engine 215 in the memory module 230 of one DPE 110 to transfer windowed data from the memory bank 220 to another DPE 110 over the interconnect 205. In yet another example, the compiler assigns a stream channel on the interconnect 205 and a stream channel on the receiving core 210 or on the receiving DMA engine 215.

[0104] In block 525, the compiler selects a synchronization technique for transferring data between the kernels. This is illustrated in Figure 9 In block 525, the compiler selects a synchronization technique for transferring data between the kernels. This is illustrated in

[0105] The compiler also handles synchronization between kernels when performing double buffering or single buffering. For example, when performing double buffering, the compiler can establish a locking protocol for accessing the double buffer 905 that can not be needed when performing single buffering (e.g., when the kernels are on the same DPE 110). In another example, the compiler can select a ping / pong synchronization technique for the double buffer 905. In any case, the synchronization can be established by the compiler using parameters provided by the programmer in the source code.

[0106] Returning to the method 500, in block 510, the compiler transmits a bitstream and / or binary code (e.g., a series of memory-mapped store transactions) to configure the SoC to execute the dataflow graph using the compiled source code. That is, the SoC can receive the bitstream / binary code and then execute the graph using the hardware elements specified by the compiler. The compiler can determine where each kernel should be placed in the SoC, the type of communication link between the kernels, and the synchronization used by the communication link.

[0107] Figure 10 A hardware view 1000 of a portion of a SoC implementing a dataflow graph in Figure 7 according to an example. That is, the hardware view 1000 illustrates a portion of a SoC implementing the dataflow graph shown in Figure 7 in block 505. In this example, the hardware view 1000 illustrates a portion of a SoC including a PL 125, and at least a portion of a DPE array including five cores 210 and five memory modules 230. Figure 10

[0108] Kernels a and f are formed using configurable logic blocks (CLBs) in the PL 125. Kernel a is communicably coupled to the memory module 230A by the interconnect 205. Although not shown, this communication link between kernel a and the memory module 230A can also include a NoC and a SoC interface block that allows the cores 210 in the DPE array to communicate with other hardware modules in the SoC (e.g., the PL 125). In this embodiment, kernel a transmits data to a DMA engine 215A in the memory module 230A, which stores the received data in a double buffer 905A in a memory bank 220A. Thus, the compiler decides to implement the communication link 620A shown in Figure 9 by assigning the double buffer 905A to the memory bank 220A. Using DMA writes, kernel a can store data in the double buffer 905A, which can then be accessed by kernel b located on core 210B.

[0109] ​In this example, double buffer 905A is allocated four banks in memory bank 220A. In an embodiment, each memory bank holds 128 bytes, which means that double buffer 905A is 512 bytes in total. However, the compiler can allocate more or fewer memory banks to double buffer 905A depending on the anticipated needs of kernels a and b. Kernel a can write data into two memory banks 220A in double buffer 905A, while kernel b reads data from the other two memory banks 220A in buffer 905A. In an embodiment, the compiler establishes a ping / pong synchronization protocol between kernels a and b such that the kernels do not attempt to access the same pair of memory banks. As described above, the compiler can handle the synchronization protocol such that kernel a on PL 125 can communicate with kernel b on core 210B using the type of communication (e.g., windowed or streaming) that the programmer has indicated in the graphical source code should occur between these kernels.

[0110] In an embodiment, because core 210B hosting kernel b is directly adjacent to memory module 230A, kernel b can access double buffer 905A directly without having to use interconnect 205 (unlike kernel a). Thus, when allocating double buffer 905A and kernel b to hardware elements, the compiler selects memory module 230A and core 210B to be directly adjacent to each other so that kernel b can use the direct connection between core 210B and memory module 230A, which has higher throughput than using interconnect 205.

[0111] Because kernels b and c are hosted or allocated to the same core 210B, as Figure 9 shown, the compiler attempts to allocate single buffer 910A to an adjacent memory module 230. In this case, the compiler allocates single buffer 910A to memory module 230C, but could have used any adjacent memory module— e.g., module 230A or 230B. The compiler can have chosen memory module 230C over modules 230A or 230B so that these memory modules have more available space for use by the more northerly cores in the array (not shown). Regardless of the reason, kernels b and c can use the direct connection between core 210B and memory module 230C to transfer data into and out of single buffer 910A. Because kernels b and c are allocated to the same core 210B and thus are executed sequentially rather than in parallel, a single buffer 910A is sufficient rather than a double buffer, because at any given time, core 210B is executing only one of the kernels. In this example, single buffer 910A includes two memory banks of memory bank 220C, but the compiler can allocate more or fewer memory banks depending on the anticipated needs of kernels b and c.

[0112] For the inter-kernel communication link between kernels c and d (shown as communication link 620C in Figure 9 FIG. 6C), the compiler allocates a double-buffer 905B to the memory banks 220B in the storage module 230B. As described above, the compiler can establish a ping / pong synchronization protocol between kernels c and d to simultaneously write and read two respective pairs of memory banks 220B in the double-buffer 905B. Moreover, by using the storage module 230B adjacent to the core 210B hosting kernel c and the core 210C hosting kernel d, the compiler leverages the direct connection these cores 210B-C have to the storage module 230B to read and store data in the double-buffer 905B.

[0113] For the intra-kernel communication link between kernels d and e (shown as communication link 620D in Figure 9 FIG. 6D), the compiler allocates a single-buffer 910B to the storage module 230C. As with the communication link between kernels b and c, a single-buffer 910B is sufficient because kernels d and e execute sequentially on the core 210C.

[0114] For the inter-kernel communication link between kernels e and b (shown as communication link 620E in Figure 9 FIG. 6E), the compiler allocates a double-buffer 905D to the remaining four memory banks 220C in the storage module 230C that are not already used by the single-buffers 910A and 910B. The compiler can again establish a synchronization protocol between kernels b and e to access the double-buffer 905D.

[0115] For the heterogeneous communication link between kernels d and f (shown as communication link 620F in Figure 9 FIG. 6F), where the kernels are hosted on different types of processing cores (e.g., a PL 125 and a DPE containing core 210C), the compiler allocates a double-buffer 905C to the memory banks 220D in the storage module 230D. Kernel d can access the double-buffer 905C by using the direct connection between the core 210C and the storage module 230D. However, because kernel f is hosted on the PL 125 instead of on a core 210, kernel f can access the double-buffer 905C by using the DMA engine 215D and the interconnect (as well as the NoC and SoC interface buffers, which are not shown). The compiler can again establish a synchronization protocol between kernels d and f to allow them to access the double-buffer 905C in parallel.

[0116] While Figure 10The illustration shows kernels that communicate with each other within the same core 210 or within cores 210 directly connected to the same storage module, placed in a DPE array. However, in other embodiments, the compiler can place two kernels on cores that do not have direct connections to the same storage module 230. That is, the compiler can assign two kernels that communicate directly in the graph to two non-adjacent cores 210. In this case, the compiler can configure the kernels to perform DMA read / write or streaming connections (similar to kernels located in PL 125) using interconnect 205 to enable communication between kernels without using shared memory.

[0117] In this way, the compiler can determine the placement of kernels in a heterogeneous system; determine the type of communication link between kernels (double-buffered, single-buffered, windowed, or streaming); and establish synchronization protocols between kernels using parameters defined by the programmer in the source code (e.g., parameters defining the communication links). However, as mentioned above, if the programmer knows in advance the optimal solution for implementing the graphics defined in the source code on the SoC, the programmer can use constraints to provide optimization instructions to the compiler.

[0118] Figure 11 An overlapping window 1100 is shown, according to an example, used when transferring data between kernels. In one embodiment, the overlapping window 1100 may be defined by a kernel (e.g., Figure 10 The kernel a) receives and forms the stream data, then divides the stream data into blocks to generate Figure 11 The overlapping window 1100 is shown. In another example, the kernel may have already received the overlapping window from the upstream kernel and then transferred it to the downstream kernel. In one embodiment, due to ping-pong synchronization, window 1100A is stored in one of the double buffers 905A-D and window 1100B is in the other buffer. The compiler is then responsible for ensuring that overlapping 1105 is copied from one buffer to the other before the next kernel call.

[0119] Overlapping windows 1100 can be useful in some embodiments, but not in others. For example, overlapping windows 1100 can be useful in the wireless domain, so that the SoC can maintain the state of the core between executing different windows. In one embodiment, after the core finishes processing a kernel, the registers associated with the kernel are cleared, so that the state of the kernel is lost. However, by providing an overlap 1105 between windows 1100A and 1100B, where the data in overlap 1105 is the same, when the core then begins processing the new data in window 1100B, the core can regain the state it had when it finished processing window 1100A. In other words, by processing overlap 1105 in window 1100B, which contains the last sample in window 1100A, the core regains the state it had at the end of processing window 1100A. The core can then begin processing the new data in window 1100B that is not in window 1100A. Thus, the block size 1110 of window 1100B indicates the new data that the core is processing that is not in the previous window 1100A. In this way, the graph can use windows 1100 (which can reduce the stall of the core with respect to the stream data) to process the received data, but still maintain the illusion of an infinite stream by using overlap 1105.

[0120] If the communication links between the kernels use windows (rather than streams), in one embodiment, the receiving kernel does not process the data until it receives all of the windows 1100 of data from its inputs, which makes processing the data non-blocking. Once all of the windows 1100 of data are received, the kernel processes the data without stalling for further data, and outputs the windows to the downstream kernel or kernels. For example, Figure 9 Kernel d in Figure 9 outputs data windows 1100 to kernels f and e in parallel using communication links 620F and 620D, respectively. The data windows 1100 output by kernel d to kernels f and e can be the same data or different data.

[0121] In another embodiment, rather than waiting until all of the windows have been received or all of the data is ready to be output, the user can program the kernel to determine when it receives input data or outputs data. For example, referring back to Figure 6 Communication link 620E in Figure 6 is asynchronous, where the source code that defines kernel b determines when it receives data from kernel e.

[0122] Returning to method 500, the control program controls the execution of the dataflow graph on the SoC. That is, once the kernels and communication links have been allocated to the various hardware components, and as described above with respect to method 400, the control program controls the execution of the dataflow graph on the SoC. Figure 10The control program can then provide instructions to the SoC to control the execution of the graph. As noted above, the control program can be executed on a host computing system, such as can be preferred in a data center, or within the PS of the SoC. In an embodiment, the control program is compiled by using control source code.

[0123] Figure 12 is control source code 430 for a control program for a dataflow graph according to an example. The source code 430 provides connections 1205 that indicate to a compiler how data should be read into and out of the graph. Major categories include control APIs for initializing the graph (e.g., init()), running the graph (e.g., run()), and ending the graph (e.g., end()). For example, a programmer can use the control source code 430 to indicate how many iterations the graph should run before stopping. This can be useful for debugging purposes. In other examples, however, the control program can allow the graph to run indefinitely according to the application. These control APIs will be discussed in more detail later.

[0124] In an embodiment, a programmer can want a large lookup table (LUT) that exceeds the storage module size. Once the compiler identifies a LUT that is too large for any storage module in the DPE array, the compiler can distribute the LUT across multiple storage modules. The compiler can allocate the LUT directly onto an array. The programmer can declare the LUT as static data and an array parameter and connect the static data and array parameter to a kernel. The compiler treats the LUT as internal data to the kernel (similar to a coefficient table). This declaration of the LUT is in the graph and is allocated as a graph component. In an embodiment, a large LUT is not double buffered and can only be accessed by one kernel at a time.

[0125] In an embodiment, a kernel can read / write a stream directly from a core in the DPE. In the kernel source code, the stream can be declared as a function parameter. If data on the stream port in the core is not available, the kernel can stall (so no locking mechanism is needed). It is a per-element synchronization by the hardware implementation of the stream itself, and although the core can stall for lack of input data available, there is no memory conflict on the storage bank, or the output buffer is full.

[0126] In an embodiment, if the kernel requires more cycle counts than any of the cores on the core can provide, it is split among the cores and a cascaded stream is used to connect the split kernels. In the source code, the programmer expresses multiple kernels that are linked together to form a cascade. The overall computation is the cumulative sum of the entire chain. The compiler distributes the computation of the cascaded kernels across multiple cores. The cores perform the per-cycle accumulation in the core's registers, i.e., using internal registers in the core without using a storage module. Thus, the cores can perform the chain using register-to-register communication without needing to use a storage module as a buffer (e.g., the single buffer and double buffer described above). In an embodiment, rather than the programmer linking multiple kernels to form a cascade, the compiler (or some other software application) can perform this conversion, where the kernel is split among the cores to form a cascade.

[0127] Constraints

[0128] Figure 13 is a flowchart of a method 1300 for compiling source code to implement a dataflow graph using constraints according to an example. In block 1305, a compiler identifies user-defined constraints in source code that establishes a dataflow graph. For example, referring to Figure 6 A programmer can add the constraints 615 to the graph source code 420. However, in other embodiments, the programmer places the constraints in the kernel source code. In other embodiments, the programmer can define the constraints in a separate file. The graph source code can reference or link to the file so that the compiler can identify the constraints when implementing the dataflow graph.

[0129] The user-defined constraints are external constraints because they are generated by the programmer rather than the compiler when compiling the source code to implement on the SoC. In an embodiment, the number of external constraints provided by the programmer can vary depending on the intelligence of the compiler. If the compiler has internal constraints that result in an optimized implementation of the dataflow graph, the programmer can choose to provide few constraints. Thus, the ability of the compiler will influence the number of external constraints that the programmer decides to use. As newer, more intelligent versions of the compiler become available, the programmer can provide fewer constraints.

[0130] The type of constraints can vary. Also, the number of constraints provided by the programmer can be related to the degree of understanding the programmer has of the underlying hardware in the SoC. If the programmer knows very little about the hardware of the SoC, the constraints can dictate the overall performance of the dataflow graph (e.g., the required performance of the dataflow graph, such as the cycle time or latency of the graph). If the programmer understands some of the underlying hardware constructs in the SoC (e.g., DPEs, PLs, communication link types, etc.), the programmer can also provide constraints for these specific graph objects. Thus, some constraints can be hardware-agnostic (e.g., performance constraints that affect the entire graph), while other constraints are hardware-aware and affect specific graph objects (or groups of graph objects) in the dataflow graph.

[0131] As an example of a hardware-aware constraint, the programmer can specify where a particular kernel should be located in the DPE array (e.g., a kernel location constraint). Or the programmer can specify a location relationship between two kernels (e.g., the two kernels should be hosted on the same core or hosted on adjacent cores). In another example, a constraint can specify where a particular buffer of a communication link (or a port of a kernel) should be placed in the DPE array. The location requirement of the buffer can be an absolute address or memory bank, or it can be a relative location with respect to another buffer or kernel or a stack associated with the processor that executes the kernel. Another type of constraint can indicate whether a particular buffer should be placed in a memory module adjacent to the core that hosts a particular kernel. Another type of constraint can apply to the entire dataflow graph. Using these types of constraints, the programmer can control how the compiler places graph objects (e.g., kernels, ports, communication links, etc.) in the SoC.

[0132] The programmer can also provide hardware-agnostic performance constraints. For example, the programmer can want the latency of the graph to be less than a certain number of processing cycles. The compiler can test its implementation of the graph to determine whether it satisfies the performance constraint, and if not, reconfigure the graph until the constraint is satisfied. For example, if two kernels were previously located on the same core, the compiler can split them to two different cores, or move the buffer to a shared memory module so that the kernels can access the data directly without using the interconnect in the DPE array.

[0133] In another embodiment, the constraints can define the utilization of a core / port / FIFO / memory module or a preferred FIFO depth. The compiler can test its implementation of the graph to determine whether it satisfies the performance constraint, and if not, reconfigure the graph. Because the compiler often tests the graph to determine whether the constraint is satisfied for performance constraints, these constraints can also be referred to as derived constraints.

[0134] In block 1310, the compiler identifies the graph object corresponding to the constraint by using the unique name in the constraint. In this example, each graph object can be assigned a unique name - e.g., each kernel, communication link, port, etc. When formatting the constraint, the programmer can use the unique name to inform the compiler which graph object the constraint applies to.

[0135] In one embodiment, the programmer can provide a unique name for each graph object in the index. The compiler can then access the index. In another embodiment, the compiler assigns the unique names to the graph objects. For example, the compiler can form a hierarchical tree of all the graph objects in the graph and assign unique names to the objects by traversing the tree from root to leaf. The programmer can also access the hierarchical tree, so she can assign a constraint to a particular object by using the unique name.

[0136] In block 1315, the compiler configures the graph objects to satisfy the constraints when compiling the source code. Various examples of placing graph objects according to constraints are shown in Figure 14

[0137] Figure 14 is a DPE array 105 according to an example, with graph objects implemented using user-defined constraints. In this example, the graph objects include kernels a-d and buffers 905. In one embodiment, the compiler places kernel a on core 210H in response to a location constraint provided by the programmer. For example, the programmer can use the unique address 1405 assigned to core 210 to instruct the compiler to place kernel a on core 210H. That is, the constraint can include the address 1405 of core 210H (i.e., 2, 1), which instructs the compiler to place kernel a on core 210H.

[0138] Figure 14 A hosting constraint 1415 is also shown, which indicates that kernels b and d should be assigned to the same core 210E. While the programmer can format the constraint in the source code to require the compiler to place both kernel b and kernel d on core 210E (e.g., using its address 1, 1), in another embodiment the constraint can not specify a particular core, leaving the compiler free to determine the optimal core 210 to host kernels b and d on its own.

[0139] Figure 14 A relative location constraint 1410 is also shown, which instructs the compiler to place kernel c and kernel b on adjacent cores - i.e., cores 210D and 210E. Again, while the programmer can format the constraint to indicate which two of the cores 210 in the DPE array 105 should host kernels c and b, in another embodiment the compiler can be free to select the cores 210 based on other metrics, such as availability. ​

[0140] Further, Figure 14 Placement of buffer 905 is shown according to constraints provided by the programmer. In an embodiment, the programmer specifies in the constraints that buffer 905 should be placed in a storage module 230B using, for example, tile address (0, 1). Alternatively, the constraints can not provide the absolute position of the storage module in array 105, but instead specify that buffer 905 is to be placed in a storage module 230 that is directly accessible by the core corresponding to kernel d. Doing so gives the compiler the freedom to choose one of the four storage modules 230 surrounding core 210E to use to implement the buffer using metrics such as availability. In another embodiment, multiple buffers can be mapped to the same memory group (e.g., the stack / reserved memory of a kernel group is mapped to the same memory group) by the constraints.

[0141] Figure 14 Only a few of the position constraints that can be used to place a graph object in DPE array 105 are shown. As described above, the programmer can provide Figure 14 Other external constraints (or other derived constraints that the compiler can recognize) not described in the above can be used to customize the graph according to the programmer's preference. Further constraint types can include the routing resources that the path should take to transfer data from one point to another; whether the data path should be circuit switched or packet switched; and how much delay should be inserted on the data path. Some constraints can help the compiler make better decisions when generating the compiled code. Other constraints can improve the performance of the SoC, such as buffer-to-buffer placement constraints to avoid memory conflicts.

[0142] Returning to method 1300, in block 1320, the compiler implements the dataflow graph in the heterogeneous processing system of the SoC according to the constraints. As described above, the compiler can generate a bitstream and binary code for configuring the heterogeneous processing system in the SoC to execute the dataflow graph.

[0143] In an embodiment, the dataflow graph can be extended to multiple SoCs (e.g., multiple FPGAs). In that case, the graph source code can include first constraints for configuring a first graph object in a heterogeneous processing system of a first SoC and second constraints for configuring a second graph object in a heterogeneous processing system of a second SoC.

[0144] Figure 15is an inheritable abstract interface 1505 according to an example. In this example, the abstract interface 1505 defines an interface for a filter chain 1510 that includes a port 1515. The interface 1505 can be defined by a programmer in different ways using software classes. For example, a filter chain 1520 inherits the abstract interface 1505 and includes kernels a and b. In contrast, a filter chain 1525 also inherits the abstract interface 1505 but includes kernels a, b, and c. For example, the filter chain 1525 can require more fine-grained processing than the filter chain 1520. Because the abstract interface 1505 can be defined using an object-oriented programming language, the interface 1505 can be inherited and used for different implementations.

[0145] Figure 16 is a dataflow graph 1600 with multiple subgraphs 1505 according to an example. Figure 16 is different from Figure 15 in that the source code for the dataflow graph 1600 includes two instances of a subgraph - namely, subgraphs 1505A and 1505B. That is, a subgraph 1505 can be defined once and multiple instances of that subgraph 1505 can be inserted into a graph 1600. For example, a receiver chain defined by the graph 1600 can use filters defined by two subgraphs 1505 because it corresponds to a dual-channel system rather than a single-channel system in Figure 15 In this way, a subgraph 1505 can be defined separately from the graph source code (e.g., in its own file) and then instantiated any number of times.

[0146] In Figure 16 , kernel b is modified to include a first port 1510B that sends data windows to the subgraph 1505A and a second port 1510A that sends data windows to the subgraph 1505B. This can be defined by a programmer in the source code.

[0147] Figure 17 is a constrained dataflow graph 1700 according to an example. Figure 17 includes Figure 16 the graph 1600 shown, which includes multiple instances of a subgraph 1505. However, the graph 1600 is contained within the constrained dataflow graph 1700. In an embodiment, the constrained graph 1700 is a wrapper graph that adds constraints to the logical design. That is, by encapsulating the graph 1600 in the constrained graph 1700 (which can be accessed using a port 1705), a programmer can add overall constraints to the execution of the graph 1600. Furthermore, a compiler can automatically propagate constraints from the constrained graph 1700, can convert the graph 1600 to a different implementation, and can then instantiate it as another dataflow graph.

[0148] Figure 18This is a constraint processing flow 1800 based on the example for merging constraints from multiple sources. Flow 1800 includes graphical source code 1805, which includes constraint 1810, which can include any of the constraint types discussed above. Furthermore, flow 1800 includes constraints 1815 from other sources, which can also include any of the constraint types discussed above. These latter constraints can be defined using JavaScript Object Notation (JSON) file format, TCL file format, or by using a graphical user interface (GUI). Therefore, constraints 1815 from other sources are not embedded within source code 1805, but are presented as separate files.

[0149] During constraint processing 1820, the compiler merges constraint 1810 from source code 1805 with constraints 1815 from other sources. In one embodiment, constraints (regardless of where they are defined) have a format that allows them to be merged with the compiler's internal data structures. In another embodiment, the programmer can specify constraints individually for each sub-graph, and the compiler can process reading these constraints and merging them with the parent graph program defined in source code 1805.

[0150] Constraint client 1825 (e.g., partitioner, mapper, and router) receives the merged constraints and ensures that solution 1830 satisfies the constraints. That is, constraint client 1825 ensures that the implementation of the data flow graph in the SoC satisfies constraint 1810 embedded in source code 1805 as well as constraint 1815 from other sources.

[0151] Control API

[0152] Figure 19 This is a block diagram of a computing system 1900 for implementing dataflow graphics on a SoC, based on an example. The computing system 1900 includes many components similar to those described above. Figure 4 The same components discussed earlier are not discussed in detail here. However, Figure 19 and Figure 4 The difference lies in that the computing system 1900 includes a control API 1905, which may or may not be present. Figure 4 In the computing system shown, as illustrated, control API 1905 is configured in control source code 430.

[0153] In general, programmers can use the control API 1905 to change parameters that control execution of the dataflow graph 440 on the SoC 100. That is, embodiments herein use the API 1905 and corresponding methods to control, interact with, and at least partially reconfigure user applications (e.g., dataflow graphs 440) executing on the heterogeneous processing system of the SoC 100, either through native control programs compiled from the control source code 430 or by executing the control source code on the PS itself. Using the control API 1905, users can directly manipulate such remotely executing graphs as native objects and perform control operations on them (e.g., for loading and initializing the graph; dynamically adjusting parameters for adaptive control; monitoring application parameters, system state, and events; scheduling operations to read and write data across distributed memory boundaries across platforms; controlling the execution lifecycle of subsystems; and reconfiguring compute resources for new subsystem parts, etc.).

[0154] For example, a kernel or other graph object in the SoC 100 can have parameters that control the operation of these objects, such as gain or filter coefficients. These parameters can be dynamically controlled using a control program executing on the host or on the SoC itself. The compiler 435 can configure the control program to change the parameters, meaning that programmers can express the API 1905 at a high level (using source code), and the compiler 435 handles the hardware details for adjusting the parameters, such as configuring registers, identifying routes, identifying locations of graph objects, etc.

[0155] Advantageously, the compiler 435 can configure drivers 1910, registers, and other hardware in the SoC 100 so that the API 1905 can perform the desired functions. For example, a driver 1910 can be used to perform a DMA to read data in a DDR memory in the SoC 100 into one of the DPEs 110 executing a kernel of the dataflow graph 440. While the driver 1910 is shown as part of the PS 130, in other embodiments, the driver 1910 can be implemented using a controller in the PL 125 or by using control signals transmitted to the SoC 100 from a remote controller using a network.

[0156] Without the control API 1905, programmers would have to directly configure the driver 1910, which can require the programmer to know the location of the kernel (e.g., host DPE) and the path to the kernel. Instead, the compiler 435 can configure the driver 1910 in response to detecting the corresponding API 1905 in the control source code 430. That is, when the API 1905 is defined, the programmer simply identifies the graph object (e.g., a particular kernel or kernel port) and the compiler 435 can do the rest—e.g., configure the driver 1910 and program registers to perform a DMA.

[0157] Figure 20A and 20B An example control API for controlling execution of a dataflow graph on a SoC is shown. Figure 20A A list of control APIs 1905 that can be used to control the operation of a dataflow graph is shown. Figure 20A Comments are included next to each API 1905 explaining its purpose. For example, the graph() API defines an empty dataflow graph class constructor. All user-defined graphs are extensions of this class.

[0158] The init() API initializes a dataflow graph; the run() API executes a graph; the wait() API waits for a previous run of a graph to complete or for a number of cycles, then suspends the graph; the resume() API resumes a graph after it has been suspended; and the end() API waits for the last run to complete, then disables the DPE. Thus, using these APIs 1905, a programmer can control when a graph starts running, how long it runs, and when the graph ends.

[0159] The update() API allows a programmer to update runtime parameters in a dataflow graph by specifying graph objects (e.g., by using the input_port &p pointer). Using the information provided, the compiler can configure the hardware in the SoC to perform the update using triggers, discussed below.

[0160] Using the read() API, a programmer can read runtime parameters from a dataflow graph that is executing. This is particularly useful for controlling graph execution based on dynamic data-dependent decisions.

[0161] Figure 20B Other control APIs 1905 that can be part of the programming model are shown. Figure 20BA global memory input / output (GMIO) class with special APIs for moving data between the DPE array and the DDR memory in the SoC is included. For example, an init() API initializes a GMIO object by providing a set of memory addresses that exist in the DDR memory. A gm2me_nb() API can transfer data from global memory to the DPE array using DMA registers in the shim. In an embodiment, the compiler configures the registers in the shim to execute the APIs 1905 within the GMIO class. Further, these APIs 1905 are non-blocking commands, which means that the PS (which can host the control program) can perform other functions concurrently with GMIO reads and writes. In an embodiment, the GMIO APIs allow the SoC to use the same set of DDR memory to transfer data into and out of the DPE array. That is, a programmer can use the GMIO APIs to read data from the DDR memory into the DPE array, which processes the data and stores the processed data in the same DDR memory.

[0162] Figure 20B A programmable logic input / output (PLIO) class with APIs for moving data between the PL and the DPE array is also included. The PLIO APIs are more direct than the GMIO APIs, as it can be used only for analog environments that transfer data between the DPE array and input / output files.

[0163] Figure 20B An event class is also included, which contains APIs for monitoring performance or executing specific graph objects (e.g., a specific kernel of a GMIO port). The event APIs allow a programmer to track specific hardware events, count occurrences of hardware events, and measure aggregate performance metrics. In an example, a programmer can measure the latency of a graph by tracking the inputs and outputs of the dataflow graph. For example, in response to an API, the compiler can set up a performance counter that counts the number of processing cycles from when a first data input enters the dataflow graph until the dataflow graph outputs the first data. In another example, a programmer can measure the throughput of a graph executing within a DPE. The compiler can set up a performance counter to count the number of cycles and the number of data items produced during a certain number of iterations of the graph execution.

[0164] Figure 21A logical partitioning of the DPE array 105 into different regions is shown according to an example. In this embodiment, the top region 2105 includes the entire DPE array 105 and its DPEs 110. The RC region 2110 includes a subset of columns in the DPE array 105. Regions 2115A and 2115B define sub-regions within the RC region 2110. In this way, the DPE array 105 can be partitioned into a hierarchy of regions. In this example, the RC region 2110 is a sub-region of the top region 2105, while regions 2115A and 2115B are sub-regions contained within the RC region 2110.

[0165] Using the APIs and constraints discussed above, a programmer can assign different dataflow graphs to different regions in the array 105. For example, multiple dataflow graphs can process digital data obtained from a radio transceiver, which can receive data using a different number of antennas depending on the time of day. To disable or enable dataflow graphs corresponding to antennas, the programmer can place each dataflow graph in a separate RC region 2110 using placement constraints, so that process control corresponding to a particular antenna can be selectively enabled and disabled. Thus, placing different dataflow graphs in different regions gives the programmer control so that one dataflow graph can be enabled or disabled without affecting dataflow graphs operating in different regions. In one embodiment, the programmer provides multiple logically independent container graphs derived from the class RC graph and assigns multiple dataflow graphs to them. The compiler then determines the specific hardware region for each container graph so that each dataflow graph can be controlled independently.

[0166] In another embodiment, the programmer can use the control APIs discussed above to establish multiple alternative graphs in a single container graph. An alternative graph is a dataflow graph that shares the same logical container graph, and thus the same hardware region. If the number of alternative graphs for a container graph is greater than 1, this means that different dataflow graphs share the same hardware region but are executed at different times. In one embodiment, the assignment of container graphs and alternative dataflow graphs to specific regions is defined in a data packet binary file provided by the compiler to the SoC.

[0167] Figure 22A Dynamic changes to the execution of a dataflow graph are shown according to an example. That is, Figure 22A The use of one or more control APIs to dynamically reconfigure a dataflow graph 2200 (e.g., change run parameters) to change how the graph 2200 processes data is shown. This reconfiguration can occur without changing the underlying hardware. That is, after the SoC is initialized, the dataflow graph 2200 can be switched between different states on the fly without needing to reconfigure the hardware.

[0168] Dataflow graph 2200 shows a processing scheme including a dedicated LTE20 lane 2205, a dedicated LTE10 lane 2215, and a reconfigurable lane 2210, which can be selectively changed between LTE20 and LTE10 lanes using a runtime parameter 2220. For example, to configure lane 2210 as an LTE20 lane, parameter 2220 controls multiplexer 2230 so that it outputs data received from a half-band filter. A control API can change parameter 2220 so that multiplexer 2230 ignores data output by the half-band filter and delay alignment block, so that lane 2210 processes data similarly to LTE10 lane 2215.

[0169] In an embodiment, multiple reconfigurable alternatives within dataflow graph 2200 can be allocated to the same region in a SoC. This is shown in Figure 22B For example, graph 2250 can be allocated to a top region 2255 in a SoC. Alternatively, different lanes in graph 2200 are allocated to different regions. In this example, instead of having reconfigurable lane 2210 including multiplexer 2230, graph 2250 is constructed using two alternatives of a reconfigurable container RC region 2260. One alternative AltO 2265 is an LTE20 lane, and the other alternative Alt 1 2270 carries two LTE10 lanes along with a mixer 2275. The fixed LTE20 lane in AltO 2265 can be allocated to its own region in the SoC, separate from the region or regions to which the two LTE10 lanes are allocated, or it can be part of the top region 2255. Thus, when RC region 2260 is supposed to be used as an LTE20 lane, a control API can reconfigure the region to load graph AltO 2265 (without affecting the dedicated LTE20 lane arranged in other regions). However, when RC region 2260 is supposed to be used as two LTE10 lanes, the control API can reconfigure the region to load alternative graph Alt 1 2270. While this avoids the circuitry for dynamically reconfiguring lane 2210, such as multiplexer 2230, shown in Figure 22A

[0170] Figure 23A 23B ​​Triggers and asynchronous parameters are shown according to examples. For example, unlike windows and streams that correspond to streaming data, parameters can be used to control execution of a dataflow graph using non-streaming data. In one embodiment, a programmer uses a synchronous trigger to change a parameter in a dataflow graph at the beginning of kernel execution. In another embodiment, the change of a parameter can occur asynchronously from the execution of a kernel. In one embodiment, a control program (whether executing on the PS or on the host) initiates a triggered change or an asynchronous change of a parameter. In another embodiment, programmable logic initiates a triggered change or an asynchronous change of a parameter. Examples of parameters that can be changed using triggers include parameters in a function or method call or parameters that change window size.

[0171] Figure 23A Parameters that are triggered are shown, where the kernel waits for new parameters each time the corresponding function is called. As a result, the kernel does not execute until the control program 2305 provides a triggered parameter. For example, the control program 2305 generates a write event 2315A to a ping buffer of parameters received by a DPE executing the kernel 2310. In response, the kernel 2310 processes data during execution block 2325A. Meanwhile, the control program 2305 is free to perform other activities during time block 2320. That is, the control program 2305 can send a triggered parameter value to the ping buffer (non-blocking) and then can perform other tasks during time block 2320.

[0172] Notably, when the kernel 2310 finishes execution block 2325A, it does not immediately begin processing more data, even if that data is available at its input. Instead, the kernel 2310 waits until a second write event 2315B is received in the pong buffer that includes a triggered parameter (which can have the same value as in write event 2315A or a different value) to execute that execution block 2325B. Once execution block 2325B is complete, the kernel 2310 again waits until a triggered parameter in write event 2315C is received to begin execution block 2325C. In this way, triggered parameters allow the control program 2305 to transmit updated parameters to the kernel 2310 before each execution block.

[0173] Figure 23B Asynchronous parameters are shown, where the kernel 2310 uses previously received parameters to execute. As shown, the control program 2305 transmits a write event 2315D to the ping buffer that includes an updated parameter to be used by the kernel 2310 when processing data during execution block 2325D. Meanwhile, the control program 2305 can perform other activities during time block 2320, as shown. However, unlike Figure 23A Figure 23A ​Differently, once execution block 2325D is complete, the kernel 2310 can begin processing data immediately during execution blocks 2325E and 2325F. Because the kernel 2310 did not receive new parameters from the control program 2305, the kernel 2310 processes the input data during execution blocks 2325E and 2325F using the same parameters used during execution block 2325D.

[0174] During execution block 2325E, the control program 2305 transmits a new write event 2315E to the ping buffer that includes updated parameters for the kernel 2310. After the write event 2315E is complete, the kernel 2310 can use the updated parameter values. Thus, when the kernel 2310 begins execution block 2325G, the kernel 2310 uses updated parameters (which can be different from the parameter values used during blocks 2325D-F). In this way, the kernel 2310 can execute continuously using the same parameters until the control program 2305 sends updated parameters to the kernel 2310.

[0175] In an embodiment, when a kernel is invoked, the compiler creates locking criteria to ensure that all data is available before the kernel begins processing the received data and that all data is ready for transmission before the output data window. However, for asynchronous communication, the graph does not have to make either of these checks, but the user can create an API that defines the criteria used when acquiring the input window to read or the window to write. In other words, the user-provided criteria define the point in time for kernel synchronization. For example, in Figure 6 the connection from kernel e to kernel b is asynchronous. Thus, kernel e can prepare the window, and then kernel b determines (by using the criteria provided by the user in the API) whether it should skip the first few frames before synchronizing with kernel b. That is, it is up to kernel b and e respectively to decide when to receive or output a window by using the criteria provided by the user.

[0176] In the foregoing, reference has been made to embodiments presented in the disclosure. However, the scope of the disclosure is not limited to the embodiments specifically described. Rather, any combination of described features and elements, whether related to different embodiments or not, is contemplated to implement and practice contemplated embodiments. Furthermore, although embodiments disclosed herein can achieve advantages over other possible solutions or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the scope of the disclosure. Thus, the foregoing aspects, features, embodiments and advantages are merely illustrative and are not considered elements of the appended claims or limiting. Except as expressly claimed, my scope is therefore intended to include all changes and modifications to the described embodiments that do not constitute departures from the spirit and scope of the disclosure.

[0177] Those skilled in the art will appreciate that the embodiments disclosed herein can be embodied as a system, method, or computer program product. Accordingly, various aspects can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit," "module" or "system." Furthermore, various aspects can take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

[0178] Any combination of one or more computer readable medium(s) can be utilized. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium is any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0179] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0180] Program code embodied on a computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0181] Computer program code for carrying out operations of aspects of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0182] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0183] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0184] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of the invention. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions marked in the blocks may occur in a non-linear order as indicated in the drawings. For example, depending on the function involved, two consecutively shown blocks may be executed substantially simultaneously, or sometimes these blocks may be executed in reverse order. It will also be noted that each block illustrated in the block diagrams and / or flowcharts, and combinations of blocks illustrated in the block diagrams and / or flowcharts, may be executed by a hardware-based dedicated system for a specific purpose, performing a specified function or action, or executing a combination of dedicated hardware and computer instructions.

[0186] While the foregoing is directed to specific examples, other and further examples may be devised without departing from its basic scope, the scope of which is determined by the appended claims.

Claims

1. A method for implementing data flow graphics in a heterogeneous processing system, characterized in that, The method includes: A heterogeneous programming environment is provided, which is used to define data flow graphs as object-oriented source code; Receives graphical source code generated in the heterogeneous programming environment, the graphical source code defining multiple kernels and multiple communication links, wherein each of the multiple communication links is coupled to a pair of corresponding kernels among the multiple kernels to form the data flow graph; and Compiling the graphics source code to implement the data flow graph in a heterogeneous processing system, wherein compiling the graphics source code includes: Based on the definitions of the plurality of kernels in the graphics source code, the plurality of kernels are assigned to the heterogeneous processing system, wherein assigning the plurality of kernels to the heterogeneous processing system includes: The first and second kernels are identified as communicatively coupled via the first of the plurality of communication links defined by the graphical source code; The first kernel is assigned to the first data processing engine in the heterogeneous processing system; and The second kernel is assigned to the second data processing engine in the heterogeneous processing system that is directly adjacent to the first data processing engine. Assign communication types to the plurality of communication links defined in the graphical source code, and A synchronization technique is selected, which uses the multiple communication links to transmit data between the multiple kernels.

2. The method according to claim 1, characterized in that, Both the first data processing engine and the second data processing engine have a direct connection to the shared storage module, wherein the method includes: A double buffer is allocated in the shared storage module for transferring data between the first kernel and the second kernel.

3. The method according to claim 1, characterized in that, Assigning the plurality of kernels to the heterogeneous processing system includes: The first and second kernels are identified as communicatively coupled via the first of the plurality of communication links defined by the graphical source code; The first kernel is assigned to the first data processing engine in the heterogeneous processing system; Assign the second kernel to the programmable logic in the heterogeneous processing system; and The second core is configured to use an interconnect to perform direct memory access to transfer data to the first core, wherein the interconnect includes data processing engine arrays of the first data processing engine interconnected with each other and connected to the programmable logic.

4. The method according to claim 1, characterized in that, Assigning the plurality of kernels to the heterogeneous processing system includes: The first kernel and the second kernel are identified as communicatively coupled via the first communication link among the plurality of communication links defined by the graphical source code. In response to determining that the first core and the second core have a combined cycle count that is less than or equal to the cycle budget of the first core in the data processing engine array of the heterogeneous processing system, the first core and the second core are clustered into the first core; and A buffer for transferring data between the first kernel and the second kernel is allocated in a storage module, wherein the storage module has a direct connection to the first kernel.

5. The method according to claim 1, characterized in that, Assigning the communication type to the plurality of communication links includes: Based on the definitions of the plurality of communication links in the graphical source code, select whether to use either streaming or windowing to transmit data for each of the plurality of communication links.

6. The method according to claim 5, characterized in that, The windowing includes dividing the received data into individual windows with predetermined or parameterized block sizes, and wherein each of the plurality of kernels configured to receive the individual windows waits until a window is received on each invocation before processing the received window. In at least one of the communication links that performs windowing, the individual window has data at the beginning that overlaps with the end of a previously transmitted window, such that the receiving kernel among the plurality of kernels that receives the individual window maintains its state.

7. The method of claim 1, characterized in that, The selected synchronization technology includes: Identify the double buffer allocated to the first of the plurality of communication links; and Configure a locking protocol so that the first kernel and the second kernel corresponding to the first communication link among the plurality of communication links can access the double buffer in parallel.

8. The method according to claim 1, characterized in that, The method further includes: Based on the compiled graphics source code, the transmission bitstream and binary code are configured, and the graphics source code configures the heterogeneous processing system to execute the data stream graphics; and The execution of the data stream graph in the heterogeneous processing system is controlled by a control program.

9. The method according to claim 1, characterized in that, The heterogeneous processing system includes a first chip and a second chip, wherein a plurality of cores are assigned to the first chip, wherein the graphics source code defines a second plurality of cores, and wherein compiling the graphics source code includes: The second plurality of cores are assigned to the second chip, wherein the second plurality of cores assigned to the second chip are configured to communicate with the plurality of cores assigned to the first chip.

10. The method according to claim 1, characterized in that, The graphics source code is independent of the hardware design of the on-chip system that forms the heterogeneous processing system, and can be implemented by a compiler on multiple different types of on-chip systems with different hardware designs.

11. The method according to claim 1, characterized in that, The heterogeneous processing system includes programmable logic and a data processing engine array, wherein at least one of the plurality of cores is assigned to the programmable logic, and at least one of the plurality of cores is assigned to a data processing engine in the data processing engine array.

12. The method according to claim 1, characterized in that, The method further includes: Encapsulating subgraphs within the dataflow graph, wherein the subgraphs are defined by graph classes separate from the graph source code; and Generate a constraint graph that adds constraints to the data flow graph and the subgraph, wherein the constraint graph serves as a wrapper for the data flow graph.

13. The method according to claim 1, characterized in that, Each of the plurality of kernels includes at least one port, the at least one port enabling each of the plurality of kernels to communicate with another kernel in the data flow graph, and wherein, in the data flow graph, each of the plurality of communication links couples a first port on a first kernel to a second port on a second kernel.

14. A host computer, characterized in that, The host includes: processor; A heterogeneous programming environment for defining data flow graphs as object-oriented source code; The graphical source code generated in the heterogeneous programming environment defines multiple kernels and multiple communication links, wherein each of the multiple communication links is coupled to a pair of corresponding kernels among the multiple kernels to form the data flow graph; and A compiler is configured to compile the graphics source code to implement the dataflow graphics in a heterogeneous processing system, wherein compiling the graphics source code includes: Based on the definitions of the plurality of kernels in the graphics source code, the plurality of kernels are assigned to the heterogeneous processing system, wherein assigning the plurality of kernels to the heterogeneous processing system includes: The first and second kernels are identified as communicatively coupled via the first of the plurality of communication links defined by the graphical source code; The first kernel is assigned to the first data processing engine in the heterogeneous processing system; and The second kernel is assigned to the second data processing engine in the heterogeneous processing system that is directly adjacent to the first data processing engine. Assign communication types to the plurality of communication links defined in the graphical source code, and The selected synchronization technology uses multiple communication links to transmit data between multiple cores.

Images