Method and apparatus for autonomous acceleration of data stream AI applications

Abstract

A method includes analyzing a data flow graph representing data dependencies between operators of a data flow application to identify multiple candidate groups of the operators. Determining whether the operators of a given candidate group are to be combined based on the characteristics of a given hardware accelerator and the operators of the given candidate group among the multiple candidate groups. After determining that the operators of the given candidate group are to be combined, retrieving executable binary code segments corresponding to the operators of the given candidate group; generating a binary code unit, the binary code unit including the executable binary code segment and metadata representing execution control flow in the executable binary code segment; and scheduling the code unit to the given hardware accelerator to execute the code unit.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. patent application serial number 16 / 570,822, filed on September 13, 2019, and entitled “METHOD AND APPARATUS FOR ENABLING AUTONOMOUS ACCELERATION OF DATAFLOW AI APPLICATIONS,” the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention relates to the field of data flow applications, and in particular to a method and apparatus for improving the performance of data flow programming applications by reducing the offloading overhead associated with offloading operators to a hardware accelerator for execution. Background Art

[0004] A dataflow application is a computer program (e.g., software) written using the dataflow programming method. Modern dataflow applications use a system of operators to maximize software compatibility and composability. Developers of modern dataflow applications use operators to create new algorithms by assembling them as building blocks of these algorithms. Dataflow graphs (see Figure 2 ) can be used to represent how data flows between different operators (ops) of an algorithm in a dataflow application. During runtime, the execution engine of the host processing unit offloads the operators in the dataflow application to different execution units, such as a central processing unit (CPU), a graphics processing unit (GPU), or other forms of dedicated hardware accelerators. In this approach, the hardware accelerators run passively, that is, they remain idle until new operators are formed and offloaded to them by the execution engine running on the host processing unit. The overhead of offloading a single operator to a hardware accelerator can be considerable, especially when the computation time of the operator is relatively short.

[0005] What is needed is a method and apparatus for reducing the offloading overhead associated with offloading operators in dataflow applications to a hardware accelerator for execution.

[0006] This background information is provided to reveal information believed by the applicant to be potentially relevant to the present invention. It is not necessarily intended, nor should it be construed, that any of the foregoing information constitutes prior art against the present invention. Summary of the invention

[0007] It is an object of embodiments of the present invention to provide a method and apparatus for improving the performance of data stream applications by reducing the offloading overhead associated with offloading operators to a hardware accelerator for execution.

[0008] According to an embodiment of the present invention, a method is provided, comprising analyzing a data flow graph representing data dependencies between operators of a data flow application to identify multiple candidate groups of the operators. Determine whether the operators of the given candidate group are to be combined based on the characteristics of a given hardware accelerator and the operators of the given candidate group among the multiple candidate groups. After determining that the operators of the given candidate group are to be combined, retrieve the executable binary code segment corresponding to the operators of the given candidate group; generate a binary code unit, the binary code unit including the executable binary code segment and metadata representing the execution control flow in the executable binary code segment; and schedule the code unit to the given hardware accelerator to execute the code unit.

[0009] In another embodiment, determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined according to whether the operators of the given candidate group are restricted to be executed on different hardware accelerators among a plurality of hardware accelerators.

[0010] In another embodiment, determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined based on available resources of the given hardware accelerator.

[0011] In another embodiment, determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined according to the number of the operators of the given candidate group.

[0012] In another embodiment, determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

[0013] In another embodiment, generating the binary code unit includes generating metadata representing input parameters and output parameters of each operator of the given candidate group.

[0014] In another embodiment, generating the binary code unit includes generating metadata representing a plurality of navigation table entries. A given entry in the plurality of navigation table entries includes a field associated with a size of the given entry and a pointer to a given executable binary code segment in the plurality of executable binary code segments.

[0015] According to an embodiment of the present invention, a computer system is provided, comprising a processor, a hardware accelerator and a non-transitory memory for storing instructions. When the processor executes the instruction, the instruction causes the processor to execute a binary code unit, the binary code unit comprising an executable binary code segment and metadata representing the execution control flow in the executable binary code segment. The executable binary code segment corresponds to an operator of a given candidate group. The operator of the given candidate group is selected in the following manner: a data flow graph representing the data dependency relationship between operators of a data flow application is analyzed to identify multiple candidate groups of the operator, and according to the characteristics of the hardware accelerator and the operator of the given candidate group in the multiple candidate groups, it is determined that the operator of the given candidate group is to be combined. The code unit is scheduled to the hardware accelerator to execute the code unit.

[0016] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators are combined according to constraints of the hardware accelerator.

[0017] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on available resources of the hardware accelerator.

[0018] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes: determining that the operators should be combined according to the number of the operators of the given candidate group.

[0019] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

[0020] In another embodiment, generating the binary code unit includes generating metadata representing input parameters and output parameters of each operator of the given candidate group.

[0021] In another embodiment, generating the binary code unit includes generating metadata representing a plurality of navigation table entries. A given entry in the plurality of navigation table entries includes a field associated with a size of the given entry and a pointer to a given executable binary code segment in the plurality of executable binary code segments.

[0022] In another embodiment, the binary code unit includes glue code.

[0023] According to one embodiment of the present invention, a non-transitory memory storing instructions is provided. When the instructions are executed by a processor, the processor executes a binary code unit, and the binary code unit includes an executable binary code segment and metadata representing the execution control flow in the executable binary code segment. The executable binary code segment corresponds to an operator of a given candidate group. The operator of the given candidate group is selected by analyzing a data flow graph representing the data dependency relationship between the operators of the data flow application to identify multiple candidate groups of the operator. According to the characteristics of the hardware accelerator and the operator of the given candidate group in the multiple candidate groups, it is determined that the operator of the given candidate group is to be combined. The code unit is scheduled to the hardware accelerator to execute the code unit.

[0024] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators are combined according to constraints of the hardware accelerator.

[0025] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

[0026] In another embodiment, determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Further features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flow chart showing a development workflow and an execution workflow using one embodiment;

[0029] Figure 2 An exemplary data flow diagram of a data flow application is shown;

[0030] Figure 3 Components of a super-op compiler provided by one embodiment are shown;

[0031] Figure 4 An accelerator hardware capable of executing a hyper operator provided by one embodiment is shown;

[0032] Figure 5 A navigation table for a super operator provided by one embodiment is shown;

[0033] Figure 6A super operator programming model provided by an embodiment is shown;

[0034] Figure 7 The configuration of the device provided by the present invention is shown.

[0035] It should be noted that throughout the drawings, like features are identified by like reference numerals. DETAILED DESCRIPTION

[0036] Embodiments of the present invention include methods and apparatus for improving the performance of dataflow applications based on operators that are scheduled to accelerator hardware for execution. Embodiments include systems that enable hardware accelerators to autonomously execute super-operators (multi-operator portions) of dataflow graphs with minimal intervention from a host processor. The dataflow application is executed on a computer system that includes a host processing unit and one or more hardware accelerators. The host processing unit controls the overall execution of the program and can schedule operators or super-operators to execute on the hardware accelerator. The scheduling of operators (or super-operators) to the hardware accelerator by the execution engine of the host processor is referred to as "offloading".

[0037] Embodiments include an architecture that includes a development environment and a runtime environment for dataflow programming applications. Users (such as application developers) use the architecture of the proposed system to develop and run dataflow applications. Based on the dataflow graph representation 102 of the application, the user uses a super operator compiler 104 (described below) to generate an output executable binary. The executable output of the compiler 104 can then be used by the runtime system to execute a program that includes offloading a super operator (group of operators) to accelerator hardware 108.

[0038] Figure 1A flowchart of a development workflow 100 and a runtime workflow 101 provided by an embodiment is shown. The development workflow 100 and the runtime workflow 101 can be executed sequentially or independently. The development workflow 100 is executed by a super operator compiler 104 and begins with: a graph analyzer 304 analyzes (110) a data flow graph representation 102 of a data flow application to analyze data dependencies between operators of the data flow graph 102. The graph analyzer identifies (112) a plurality of candidate groups 308 of operators that can be stored in a super operator candidate repository (repository, repo) 306. Determine (114) the group of operators to be combined to obtain input from a logic accelerator resource model 302, which includes parameters of capabilities or constraints of one or more hardware accelerators 108 available to the system. An executable binary code segment is generated by a super operator code generator 312 and can be stored in an operator binary repository (opsbin repo) 316 for future use. If the super operator code generator 312 determines that the executable binary code segment already exists, the executable binary code segment can be obtained from the operator binary repository 316 without recompiling the executable binary code segment. The super operator code generator 312 then generates (116) a binary code unit. The binary code unit includes all necessary binaries, navigation tables 314, metadata, parameters, etc. required by the runtime system to execute the data flow program (including offloading to the hardware accelerator 108).

[0039] The binary code unit can be used by the runtime workflow 101 for immediate execution or for execution at a later time. The runtime workflow 101 includes a runtime system that schedules the code unit to a host processor for execution. As part of the host processor execution, the runtime system can then offload operators and super operators to the hardware accelerator 108 according to the instructions of the binary code unit.

[0040] Figure 2An exemplary data flow graph 102 for a data flow application compiled by the super operator compiler of the present invention is shown. The exemplary data flow graph 102 includes six operators (hereinafter referred to as ops): Op1, Op2, Op3, Op4, Op5 and Op6. Operators such as Op5 are linked by edges of the data flow graph 102 representing data paths. Given an operator-based computation represented as a data flow graph 102, subgraphs that can be combined into a single operator are identified. The parameters required as part of the subgraph computation are then analyzed. The parameters may include data types, tensor shapes, data formats, and any other hyperparameters specific to individual operators within the subgraph. The values ​​of the parameters of the subgraph are used to generate code that represents the computation of the entire subgraph as a large operator, referred to herein as a super operator. The super operator can be offloaded to a hardware accelerator 108. The super operator is computationally equivalent to many basic small operators that are offloaded to the hardware accelerator separately, but the offloading overhead of the super operator is much smaller. It should be understood that although Figure 2 The exemplary dataflow graph shown in includes six operators, but a dataflow graph may include any number of operators.

[0041] Figure 3 104 is a super operator compiler provided by an embodiment of the present invention. The super operator compiler 104 includes a computing system (such as computing system 700 (see Figure 7)) processor. The super operator compiler 104 includes a logic accelerator resource model 302, a graph analyzer 304, a super operator candidate repository 306, a super operator code generator 312, and an operator binary repository 316 (hereinafter referred to as the ops bin repository 316). The graph analyzer 304 of the super operator compiler receives the data flow graph 102 of the data flow application, analyzes the data flow graph 102, identifies the operators in the data flow graph 102 that can be combined into super operators, so that they can be passed to the super operator code generator 312, and outputs super operator candidates including operators that can be combined. The graph analyzer 304 can use information and constraints from the logic accelerator resource model 302 and various other sources when making such identification. The logic accelerator resource model 302 includes information about the available resources of the hardware accelerator 108 and the constraints of the hardware accelerator 108. The data flow graph 102 may include information indicating which operators can be offloaded to the hardware accelerator (including if there is more than one hardware accelerator, indicating which hardware accelerator is available), or indicating whether offloading is allowed. The programmer may explicitly or implicitly provide a decision about operator offloading (referred to as an offloading decision). The offloading decision may also be determined by an offloading analyzer (not shown), which indicates the operators that can be offloaded. The graph analyzer 304 analyzes the data flow graph to determine the feasibility and performance cost of scheduling (i.e., offloading) each operator or group of operators. An example of a constraint is that two operators that are specified to be scheduled (i.e., offloaded) to different hardware accelerators cannot be combined. Other constraints include whether the target hardware accelerator has sufficient on-chip resources (e.g., computing power, memory size, bandwidth, etc.) to efficiently run the intended super operator. The cost model also takes into account heuristic factors such as the number of operators within a super operator, the computational requirements of the operators, the number of super operators obtained in the final graph, and the execution dependencies between operators within a super operator to balance the super operator benefits and parallel opportunities of the system.

[0042] In some cases, the super operator compiler can significantly reduce the offloading overhead. Therefore, the offloading analyzer (not shown) and the super operator graph analyzer 304 can use the common heuristics between the two to determine their respective outputs. In some embodiments, the user is also able to provide instructions indicating which operators or subgraphs to be combined. In these embodiments, the user only needs to specify which subgraph to be combined, without providing a fused implementation for the subgraph. Operators are combined based on a "shallow fusion" approach, in which existing implementations of operators are taken as is and then combined together to build a super operator. Using the shallow fusion approach, it is not necessary to create a new implementation of the fused operator with additional constraints.

[0043] The super operator candidate repository 306 receives super operator candidates from the graph analyzer 304 and stores the valid data flow graph structure defining which operators 310 are to be combined. The nodes in these sub-graphs 308 are the same operators as those from the original data flow graph 102 (eg, Op5 202).

[0044] The super operator code generator 312 receives candidate super operators (i.e., combined operator representations 308) from the super operator candidate repository 306 and generates a new super operator that is computationally equivalent to the subgraph of the individual operators used to create the super operator. It should be noted that the generation of the new super operator is systematic and does not require the programmer to provide a new implementation for the combined operator to generate the super operator. Additional metadata can be generated and attached to the new super operator to debug and analyze the calculations of the individual operators Op4, Op5, and Op6 within the candidate super operator 308. This allows users to debug and analyze super operators as if these super operators were individual operators in the data flow graph 102.

[0045] The super operator code generator 312 generates new host source code indicating which operators have been combined. The super operator code generator 312 also generates a navigation table 314 that includes a parameter list representing the parameters of all individual operators, as well as additional metadata that enables the accelerator 108 to navigate from one set of calculations of an operator to the next. The super operator 318 newly generated by the super operator code generator 312 is appended to the list of accelerator library operators. At run time, the runtime system of the host processing unit selects the super operator 318 to offload to the hardware accelerator, rather than offloading individual operators individually.

[0046] In order for the super-operator compiler to systematically generate super-operators, the (basic) operators are represented in a 'basic operator intermediate representation (IR)' format, which helps the graph analyzer 304 determine whether the operator can be combined with other operators into a super-operator.

[0047] The operator binary repository (repo) 316 stores existing target-specific implementations of super operators. For example, each super operator has its target-specific implementation as a precompiled shared object file. Any other application for the same architecture as the shared object can include and link the precompiled shared object file during compilation. The repository 316 can also store the source-level implementation of the super operator and the operators it contains. This supports flexible recompilation into binary using different compiler optimizations.

[0048] A navigation table 314 is provided to enable the accelerator 108 to perform calculations equivalent to multiple operators using a single offloaded super operator. The super operator uses an encapsulated data format. The navigation table 314 is used for data as a navigation guide for the super operator so that it can provide the necessary parameters to each operator, as well as the control flow from one operator to the next. A parameter table is generated that stores a data structure of the input and output parameters of the operators within the super operator. The data structure contents are populated by the host. In addition to the stored input and output parameters, the navigation table 314 also contains metadata indicating the control flow from the current operator to the next operator to be called within the super operator. In some embodiments, the metadata includes function pointers to the operators themselves. Figure 4 As shown, this information is provided to the hardware accelerator 108. The hardware accelerator 108 receives the super operator including the header code 402, operators 404, 406 and 408, and the navigation table 408, performs the operations and returns any resulting data as indicated. The operator binary repository 316 stores the binary of each operator in the super operator, and the super operator code generator 312 provides the super operator header code plus the operator binary to the runtime system of the host processor.

[0049] As part of the offloading process, individual operators (called basic operators) receive the provided parameters required for their calculations. In order to facilitate the automatic generation of super operators, the implementation of the basic operators must be represented in an intermediate representation (IR). Associated with the IR is a compiler that can generate separate host and accelerator hardware 108 code from the IR of the basic operator. The accelerator hardware 108 code includes a kernel function that implements an operator or super operator on the accelerator hardware 108. The compiler will define the input parameters and data in the basic operator based on the navigation table 314 information, thereby combining the host and accelerator hardware 108 codes of all basic operators in the super operator. The IR enables the compiler to generate binary operator code that can be executed by the accelerator hardware 108 and host code executed by the host processor to prepare the parameters of the binary operator code and call the binary operator code on the accelerator hardware 108. This is done separately for individual operators, and supports the definition of input parameters in the host, and supports updating the binary operator code according to the way the basic operator is combined with other basic operators within the super operator. In some embodiments, the IR consists of interface descriptors that specify the properties of the input and output data of the basic operator kernel functions on the device side. In some embodiments, the input and output data of the basic operator kernel functions are specified as function parameters and must be referenced by symbolic names in the kernel function (i.e., no hard-coded memory addresses). In some embodiments, the body of the kernel function cannot have any side effects other than those specified on the interface descriptors. The IR can represent limited forms of arithmetic operations, variable access, and control flow. It also supports calling external functions (i.e., functions that are not implemented in the IR) as long as the interface properties of the basic operator kernel functions are not violated.

[0050] In an embodiment, a super operator contains the equivalent of multiple operators. When a super operator is offloaded to a hardware accelerator, the super operator includes a navigation table of parameters 314. The navigation table of parameters 314 includes the parameters required for each individual primitive operator and additional metadata that helps the super operator determine the order of operators for which function calls must be made.

[0051] Figure 5 The structure of the navigation table 314 is shown. The navigation table 314 has a base address 502 and sub-tables 504 and 506 for each operator in the super operator. The super operator code knows which sub-table 504 or 506 the data should be retrieved from. Referring to the op1 sub-table 504, the header parameter entry size 508 describes the total size of each sub-table 504. With this information, the super operator can calculate the offset from one sub-table 504 to traverse to the next sub-table 506.

[0052] After calculating the offset loaded in the contiguous sub-table, the super operator can read the entry in the sub-table indicating the operator to be called as a function (e.g., a function pointer). The function pointer is called with the parameter entries of the sub-table as input for the function parameters.

[0053] Conventional programming of the host processing unit would require the hardware accelerator 108 to incur costly communications between the host processing unit and the hardware accelerator 108 in order to figure out the next set of computations that should be performed. The navigation table 314 relaxes this constraint and enables the hardware accelerator 108 to self-learn the subsequent computational tasks it should perform without having to iteratively communicate with the host processing unit to obtain the next set of instructions.

[0054] The process of offloading operators to hardware accelerators incurs a significant amount of offloading overhead, which is difficult to reduce due to the hardware design constraints of hardware accelerators. A super operator is a single operator that produces the equivalent of many individual operators' computations while incurring only one offloading overhead. The function calls of the super operator to multiple operators are within the hardware accelerator itself, so the offloading overhead is much smaller than the overhead associated with the communication between the host processing unit and the hardware accelerator.

[0055] The embodiments are not limited to any particular data flow framework. The embodiments may also be used for neural network applications using data flow programming techniques. Neural network frameworks that may implement the embodiments include TensorFlow, PyTorch, etc. In addition, the embodiments may be applied to different variants of hardware accelerators using a host-device offload model, such as GPUs, DSPs, FPGAs, etc.

[0056] Figure 6 A programming model 600 is shown, and how operators are offloaded to the accelerator hardware 108 by code running on a host processor, referred to as host code 602. The host code 602 represents the execution of operators that are determined to be offloaded based on the operators. A super kernel 604 includes a prolog 606 for initializing the hardware accelerator 108. The prolog 606 is followed by a first traditional kernel call 608. Glue code 610 including synchronization, navigation tables, etc. is provided between kernel calls. By reducing the number of times the host processing unit needs to communicate with the hardware accelerator 108 during runtime, the end-to-end execution time of the data flow application is shortened.

[0057] The following describes a systematic method for generating super operators provided by an embodiment. First, a target-specific function object describing operator features is created. The target-specific function object includes an operator type, input / output shape, data type, hyperparameters, and a target architecture. Next, an input / output data stream with a uniquely identified placeholder is described. The input / output data stream has a variable function estimated at runtime. These placeholders are bound to the function object. The process is iterated in each operator.

[0058] As part of the heuristic, a subset list of all function objects representing the operator that can be combined into a super operator is selected. Other variables in the heuristic include a cost-effective model of the uninstall overhead time that occurs. One criterion is that all function objects must belong to the same target architecture. Call a synthesis API that receives a list of function objects that will return a new function object representing a super operator. This will combine all metadata information contained in each individual function object, which is described above about navigation table 408.

[0059] The newly returned function object can then be called as a function, offloaded to the device, and incurs the host-device runtime overhead once per hyperoperator.

[0060] Figure 7 It is a block diagram of a computing system running a software compiler that executes the method disclosed herein. In some aspects, functions can be executed in multiple computing systems across multiple geographic locations. Computing system 700 can provide an example of computing hardware used by application developers to develop software provided by embodiments. Examples of this computing hardware can include hosting and running super operator compiler 104 and runtime system. It can also be used to execute software provided by embodiments. Computing system 700 can act as a host system to schedule super operators to accelerator hardware 108. Accelerator hardware 108 is coupled to host processing unit 700 through multiple modules. The accelerator hardware 108 can be coupled to the same wafer as the computing system as a core. It can be a separate wafer in a multi-chip package integrated circuit (integrated circuit, IC). It can be a separate IC on the same PCB as the computing system. It can be a separate module or card in a computer chassis. It can also be installed separately, close to the computer system, or away from the computer system. Accelerator hardware 108 can be coupled to IC or multi-chip modules, I / O interface 760, such as SATA interface, USB interface or thunderbolt interface, through any number of modules (including internal bus). It may also be coupled using wired or wireless techniques through the network interface 750. The accelerator hardware 108 may be coupled to the host computer system 700 using a number of modules.

[0061] Specific embodiments may use all of the components shown or only a subset of the components, and the degree of integration between computing systems may vary. In addition, a computing system may contain multiple instances of components, such as multiple processing units, processors, memories, transmitters, receivers, etc. The computing system 700 typically includes a central processing unit (CPU) 710, a bus and memory 720, and may optionally also include a mass storage device 730, a video adapter 740, and an I / O interface 760 (each shown in dashed lines to indicate that they are optional). The computing system may also include one or more network interfaces 750 for connecting the computing system to a communication network 755.

[0062] The CPU may include any type of electronic data processor and may include one or more cores or processing elements. The memory may include any type of non-transient system memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or a combination thereof. In one embodiment, the memory may include a ROM used at startup and a DRAM used to store programs and data when executing a program. The bus may be one or more of any type of several bus architectures, including a memory bus or memory controller, a peripheral bus, or a video bus.

[0063] The mass storage device may include any type of non-transient storage device for storing data, programs, and other information and enabling the data, programs, and other information to be accessed via a bus. For example, the mass storage device may include one or more of a solid state drive, a hard disk drive, a magnetic disk drive, or an optical disk drive.

[0064] Video adapter 740 and I / O interface 760 provide optional interfaces to couple external input and output devices to the processing unit. Examples of input and output devices include a display 770 coupled to video adapter 740 and an I / O device 780 coupled to I / O interface 760, such as a touch screen. Other devices can be coupled to the processing unit, and more or fewer interfaces can be utilized. For example, a serial interface (not shown) such as a universal serial bus (USB) can be used to provide an interface for external devices. Alternatively, the computing system can rely on one or more network interfaces to connect to one or more mass storage devices, one or more video adapters, and one or more I / O interfaces available on a network.

[0065] Unless defined otherwise, all technical and / or scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art.

[0066] Through the description of the above embodiments, the present invention can be implemented using only hardware, or it can be implemented using software and necessary general hardware platforms. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transient storage medium, which can be a compact disk read-only memory (CD-ROM), a USB, a flash drive, or a mobile hard disk. The software product includes many instructions that enable a computer device (a personal computer, a server, or a network device) to perform the method provided in the embodiments of the present invention. For example, such execution may correspond to a simulation of the logical operations described herein. The software product may additionally or alternatively include the number of instructions that enable a computer device to perform the operations provided in the embodiments of the present invention for configuring or programming a digital logic device.

[0067] Although the present invention has been described with reference to specific features and embodiments of the present invention, it is apparent that various modifications and combinations of the present invention may be made without departing from the present invention. The specification and drawings are therefore only to be regarded as an illustration of the present invention as defined by the appended claims and any and all modifications, variations, combinations or equivalents falling within the scope of this specification are contemplated.

Claims

1. A method performed by a software compiler, characterized in that The method comprises: analyzing a data flow graph representing data dependencies between operators of a data flow application to identify a plurality of candidate groups of the operators; determining, based on characteristics of a given hardware accelerator and operators of a given candidate group among the plurality of candidate groups, whether the operators of the given candidate group are to be combined; After determining that the operators of the given candidate group are to be combined: retrieving an executable binary code segment corresponding to said operator of said given candidate group; Generate a binary code unit, the binary code unit including the executable binary code segment and metadata representing an execution control flow in the executable binary code segment; The binary code unit is dispatched to the given hardware accelerator to execute the binary code unit.

2. The method according to claim 1, characterized in that Determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined according to whether the operators of the given candidate group are restricted to be executed on different hardware accelerators of a plurality of hardware accelerators.

3. The method according to claim 1 or 2, characterized in that: Determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined based on available resources of the given hardware accelerator.

4. The method according to claim 1 or 2, characterized in that: Determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined according to the number of the operators of the given candidate group.

5. The method according to claim 1 or 2, characterized in that: Determining whether the operators of the given candidate group are to be grouped together includes determining whether the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

6. The method according to claim 1 or 2, characterized in that: Generating the binary code unit includes generating metadata representing input parameters and output parameters of each operator of the given candidate group.

7. The method according to claim 1 or 2, characterized in that: Generating the binary code unit includes generating metadata representing a plurality of navigation table entries, wherein a given entry of the plurality of navigation table entries includes a field associated with a size of the given entry and a pointer to a given executable binary code segment of a plurality of executable binary code segments.

8. The method according to claim 1 or 2, characterized in that: The binary code unit includes a glue code.

9. A computer system, characterized in that: include: processor; Hardware accelerators; a non-transitory memory for storing instructions that, when executed by the processor, cause the processor to: executing a binary code unit, the binary code unit comprising an executable binary code segment and metadata representing an execution control flow in the executable binary code segment, the executable binary code segment corresponding to operators of a given candidate group, the operators of the given candidate group being selected by: analyzing a data flow graph representing data dependencies between operators of a data flow application to identify a plurality of candidate groups of the operators, and determining that the operators of the given candidate group are to be combined according to characteristics of the hardware accelerator and the operators of the given candidate group among the plurality of candidate groups; The binary code unit is dispatched to the hardware accelerator to execute the binary code unit.

10. The computer system according to claim 9, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators are combined according to constraints of the hardware accelerator.

11. The computer system according to claim 9 or 10, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on available resources of the hardware accelerator.

12. The computer system according to claim 9 or 10, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined according to the number of the operators of the given candidate group.

13. The computer system according to claim 9 or 10, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

14. The computer system according to claim 9 or 10, characterized in that: Generating the binary code unit includes generating metadata representing input parameters and output parameters of each operator of the given candidate group.

15. The computer system according to claim 9 or 10, characterized in that: Generating the binary code unit includes generating metadata representing a plurality of navigation table entries, wherein a given entry of the plurality of navigation table entries includes a field associated with a size of the given entry and a pointer to a given executable binary code segment of a plurality of executable binary code segments.

16. The computer system according to claim 9 or 10, characterized in that: The binary code unit includes a glue code.

17. A non-transitory memory, characterized in that: Instructions are stored therein, and when the processor executes the instructions, the processor: executing a binary code unit, the binary code unit comprising an executable binary code segment and metadata representing an execution control flow in the executable binary code segment, the executable binary code segment corresponding to operators of a given candidate group, the operators of the given candidate group being selected by: analyzing a data flow graph representing data dependencies between operators of a data flow application to identify a plurality of candidate groups of the operators, and determining that the operators of the given candidate group are to be combined according to characteristics of a hardware accelerator and the operators of the given candidate group among the plurality of candidate groups; The binary code unit is dispatched to the hardware accelerator to execute the binary code unit.

18. The non-transitory memory according to claim 17, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators are combined according to constraints of the hardware accelerator.

19. The non-transitory memory according to claim 17 or 18, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

20. The non-transitory memory according to claim 17 or 18, characterized in that: Determining that the operators of the given candidate group are to be grouped together includes determining that the operators should be combined based on a computational runtime associated with executing all of the operators of the given candidate group.

Images