Cross-hardware operator concurrent scheduling method and system

Abstract

The invention discloses a cross-hardware operator concurrent scheduling method and system. The method comprises the following steps of: 1) abstracting different operations of a target program into operators with a uniform interface, and assigning a software and hardware platform for executing the operator for each operator; abstracting operators and input and output data of the operators into objects respectively; 2) determining a dependency relationship among the objects based on the data flow of the target program, organizing the objects in a directed and looped graph mode to obtain a directed and looped graph as an abstract expression of the target program; a driver is arranged on each software and hardware platform and used for executing an operator; 3) storing nodes corresponding to each operator according to the directed and looped graph, wherein each node comprises an operator, an input object, an output object and a software and hardware platform corresponding to the operator; and 4) traversing the stored nodes by the scheduler, acquiring the currently executable nodes, concurrently scheduling operators in the currently executable nodes to corresponding hardware for execution, and receiving return values of the operators.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of program transplantation and automatic optimization, and particularly relates to a cross-hardware operator concurrent scheduling method and system. BACKGROUND

[0002] Currently, the development of computer instruction sets presents a diversified pattern. Foreign ones include the x86 architecture and x64 architecture of Intel and AMD, the ARM (Advanced RISC Machines) architecture of ARM Company, and the Power architecture of IBM based on the RISC (Reduced Instruction Set Computer) instruction set, etc. Domestic chips also develop rapidly, and self-owned intellectual property products such as the Loongson series processor (based on the RISC licensed core technology), the Godson series (based on the Alpha architecture developed by DEC), and the Feiteng processor (ARM architecture localization with self-owned intellectual property) are developed. In addition to CPUs, GPU and FPGA (Field Programmable Gate Array) boards are also widely used in scientific computing and high-performance computing fields. Each computing platform has its unique program design language, such as the use of CUDA by the GPU of NVIDIA Company, the use of ROCm platform by the GPU of AMD Company, and the use of their own exclusive programming language and environment by different FPGA products. These diversified programming environments undoubtedly increase the challenges of scientific computing personnel in programming and optimization.

[0003] In the field of scientific computing and high-performance computing, the running computing efficiency of a program is a core element. However, the complexity of the programming environment and the significant differences between hardware make it extremely challenging to write a program with excellent performance or to optimize an existing program. Therefore, program developers must develop programs specifically for a particular processor to adapt to its unique design features, ensure that the program can fully utilize the performance of the hardware, and achieve the optimal operation effect. The traditional optimization method leads to poor reusability of the program, and the original program is optimized specifically for a particular hardware, using some hardware-specific primitives, which cannot be reused on a new hardware platform. Even if part of the code can run successfully on a new hardware platform, due to the differences in hardware architecture, it is difficult to achieve performance portability, and to fully utilize the performance of the new architecture, special optimization is still needed.

[0004] The difficulty of porting the original program on the new hardware platform lies in the fact that the developer must be very familiar with the hardware architecture and make the program fully utilize the hardware performance. The conventional method is that the developer manually adjusts the timing and granularity of data transmission and calculation according to the program and hardware characteristics, so as to overlap calculation and data transmission, make full use of the processor and bandwidth, and reduce the running time of the program. Automatic implementation of the mutual overlap of calculation and data transmission not only needs to implement the concurrent execution of operators, but also needs to extract the concurrency and parallelism between operators, and in many hardware architectures that only support single-threading such as some accelerators, concurrent execution function cannot be realized through the system level.

[0005] Coroutines, also known as microthreads, are a kind of user-level lightweight thread. Coroutines do not increase the number of threads, but realize the concurrent execution of coroutines through time-sharing multiplexing based on threads. Compared with threads and processes, coroutines do not need to switch to the kernel mode, and have smaller scheduling overhead. Moreover, since it does not involve the underlying characteristics of the operating system, processors or accelerators that only support single-threading can also run successfully.

[0006] Dataflow is a kind of computing technology opposite to control flow. From the perspective of computer hardware architecture, a dataflow computer is mainly composed of dataflow nodes, which execute corresponding instructions when the required operands are ready, and the execution results are directly passed to other nodes that need the results. In a dataflow computer, instructions not only contain the input node identifier and output node identifier of data, but also the data dependency relationship between instructions, which determines the execution order of program instructions. From the perspective of software architecture, dataflow is regarded as a software paradigm, the core idea of which is to divide the computing process into multiple stages that can be executed simultaneously.

[0007] With the rise of parallel computer architecture, in order to fully utilize the performance of multi-core processors, researchers have proposed a dataflow programming model. This model can significantly improve the utilization of computing resources due to its natural parallelism. In the dataflow programming model, computation and communication are separated, and the application is described in the form of a dataflow graph. The dataflow graph describes the flow of data in the application in detail, where the nodes represent computing units and the edges represent the path of data transmission. Through the analysis of the dataflow graph, the dataflow programming model can accurately depict the dependency relationship between data, and schedule and allocate tasks accordingly. At the same time, by using software pipelining technology, the model can deeply mine the parallelism of the program.

[0008] Currently, there are many programming languages that adopt the data flow programming model, such as the StreamIt stream programming model for specific fields, which is mainly applied to scientific computing and coding and decoding processing fields. In addition, CUDA of Nvidia is also a high-performance computing programming model for GPU, which draws on the concept of the data flow model. With the wide application of big data, stream programming also plays an important role in big data systems, such as the MapReduce, Spark, Flink frameworks, which all adopt the idea of stream programming. However, the existing data flow programming models are all for specific fields or specific hardware platforms, and cannot be used across hardware platforms, and cannot solve the problem of automatic scheduling and allocation of operators.

[0009] Currently, there is no practical solution that combines control flow graphs and data flow graphs uniformly, and most of them use abstract descriptions of directed acyclic graphs or even linear queues, so there is a lot of room for development in the theory and practice of concurrent scheduling. SUMMARY

[0010] The present application proposes a cross-hardware operator concurrent scheduling method and system, hereinafter referred to as DDQ (Diao Du Qi), which can simplify the development of programs, especially the development and optimization of scientific computing programs on domestic heterogeneous processors. The present application abstracts different operations of a program as operators, especially separates data transmission and calculation into different operators, then combines the operators and data into a directed cyclic graph according to the program, thereby discovering the data dependency and parallelism therein, and then automatically scheduling the operators in the data flow graph using coroutine technology to realize concurrent execution of calculation operators and data transmission operators, making full use of processor and bandwidth resources. At the same time, this process also realizes the separation of operators and scheduling, and different operators can be called for different hardware platforms, thereby realizing the portability and reusability of programs.

[0011] DDQ is characterized by: 1) performance portability, since the operator library is separated from the core scheduler code, by calling different operators optimized for different hardware on different hardware, the program can be ported to different hardware with high performance; 2) code reuse, except for the hardware driver part, the rest of the code is general code, which can realize code reuse on different hardware architectures; 3) coroutine-based operator concurrent scheduling, operators are concurrently executed in user mode through time-sharing multiplexing, reducing the additional overhead of operator scheduling, and can be executed on most hardware; 4) based on data flow, data and operators are abstracted as objects, and the data input and output of the operator is described as the dependency relationship between objects. The state of the object can be quickly judged whether the operator can be executed, so as to improve the utilization of the processor and realize the speed optimization of the program; 5) adopt a directed cyclic graph, and the directed cyclic graph composed of objects as vertices is used as the abstract expression of the program, and the dependency relationship between objects is the directed edge of the graph. The object is stateful, and the state transition of the event-triggered finite state machine in different stages of the operator is triggered. Compared with the directed acyclic graph expression adopted by other systems, it has stronger expression ability and is more suitable for runtime scheduling, and can represent large-scale scientific computing programs containing complex control flow; 6) a script language is used to describe the directed cyclic graph, and a read-write friendly script language is provided for users, which is convenient for use and development of the operator library.

[0012] The technical scheme of the present application is:

[0013] A cross-hardware operator concurrent scheduling method, comprising the steps of:

[0014] 1) different operations of a target program are abstracted into operators with a unified interface, and each operator is assigned a software and hardware platform for executing the operator; the operator and its input and output data are abstracted into objects;

[0015] 2) the dependency relationship between objects is determined based on the data flow of the target program, and the objects are organized in a directed cyclic graph to obtain a directed cyclic graph as an abstract expression of the target program; the directed cyclic graph includes the dependency relationship between the operators and the data, and the correspondence relationship between the operators and the software and hardware platforms; a driver is set on each software and hardware platform for executing the operator;

[0016] 3) each node corresponding to the operator is stored according to the directed cyclic graph, and each node includes an operator, its input object, output object and the corresponding software and hardware platform of the operator;

[0017] 4) the scheduler traverses the stored nodes to obtain the nodes that can be executed at present; then the operator in the node that can be executed at present is concurrently scheduled to the corresponding hardware for execution by using the coroutine technology, and the return value of the operator is received.

[0018] Further, the node corresponding to each operator is stored by using a ring link list, a ring queue or a state machine model.

[0019] Further, the scheduler determines whether the node corresponding to the operator can be executed according to the state of the operator itself, the state of the input object and the state of the output object; when the operator itself and the input object are readable and the output object is writable, or when the data object of the operator is partially written, the node corresponding to the operator is an executable node.

[0020] Further, the state of the object includes: to be allocated, to be constructed, to be destructed, to be read, reading, to be written, writing, and packed.

[0021] Further, the hardware driver is an abstract interface independent of hardware.

[0022] Further, in step 1), the operator and the directed cyclic graph are described by using a script language; wherein the call of the operator, the input and output data, the data type and the attribute are specified by using the script language; wherein the input and output data of the operator are specified by using “<” and “[]”, “<” represents a directed edge in the directed cyclic graph, the data is contained in “[]” with spaces as a separator, and the data variable is specified by using a variable name or a variable type; the type of the variable is called by using a load_type primitive and a file path of the variable type; when the operator is called, the data type matching check is performed by using the context of the directed cyclic graph script and the operator script; “=” represents an equal relationship, and the data on the left and right sides of “=” is equivalent, and has reflexivity, symmetry and transitivity.

[0023] Further, in the script language, the script node is set as a syntax element of the script, the syntax structure is a description of various relationships between the script nodes, and the relationships include a binding relationship, a dependency relationship, an equal relationship, a hierarchical relationship and an operation relationship; wherein the binding relationship includes: binding a name for the script node, binding a built-in object for the script node, reading an object, and an object list; the dependency relationship includes: an input and output relationship of the operator object and the data object; the equal relationship includes: the same script node and the same binding object; the hierarchical relationship includes: script node replication and attribute relationship; and the operation relationship includes: pattern matching of a list, built-in function function, and subgraph call.

[0024] Further, the operator is a function with one return value and two void** type parameters; the return value includes: task completion and retention of the operator, task completion and destruction of the operator, task error, and temporary transfer of control right; when the return value of the operator is task completion, the scheduler obtains the node to be destructed from the ring link list and destructs the node.

[0025] The application discloses a cross-hardware operator concurrent scheduling system, which is characterized by comprising an operator library, an operation abstracting unit, an operator directed graph unit, a ringed list generating unit and a scheduler.

[0026] The operation abstracting unit is used for abstracting different operations of a target program into operators with a unified interface respectively, and assigning a software and hardware platform for executing each operator to each operator.

[0027] The operator library is used for storing the operators.

[0028] The operator directed graph unit is used for determining the dependency relationship between objects based on the data flow of the target program, organizing each object in a directed cyclic graph mode to obtain a directed cyclic graph as an abstract expression of the target program, wherein the directed cyclic graph comprises the dependency relationship between each operator and data and the correspondence relationship between the operator and the software and hardware platform, and a driver is arranged on each software and hardware platform to execute the operator.

[0029] The ringed list generating unit is used for storing nodes corresponding to each operator according to the directed cyclic graph, wherein each node comprises an operator, input objects, output objects and the software and hardware platform corresponding to the operator.

[0030] The scheduler is used for traversing the stored nodes to obtain nodes that can be executed currently, and then scheduling the operators in the nodes that can be executed currently to corresponding hardware for execution by using the coroutine technology, and receiving the return value of the operators.

[0031] Further, the scheduler determines whether the corresponding node can be executed according to the state of the operator in the node, the state of the input object and the state of the output object; when the operator and the input object are readable and the output object is writable, or the data object of the operator is partially written, the node corresponding to the operator is a node that can be executed.

[0032] The application mainly comprises program and data abstraction, dependency relationship directed cyclic graph, script language design, directed cyclic graph state machine and software and hardware driver implementation.

[0033] (1) Program and data abstraction

[0034] To implement the operatorized scheduling of a program, first the different operations of the program need to be abstracted into operators with a uniform interface. An operator is a computational unit that contains input, output and computation. In the programming language C, an operator can be implemented as a function that takes two void** arguments, which are used to represent input and output. An operator can also be described by a script that assembles existing operators into a new one. At runtime, an operator is wrapped in a memory object as a generic pointer or a structure.

[0035] Any kind of program or data can be abstracted as an object. The type, storage or access method, inheritance or containment relationship of data can be restricted by a script to implement objects for specific purposes. There is no essential difference between a program object and a data object, so an object can be generated as data and then run as a program. This feature can be used to implement high-order functions, dynamic multi-dispatch and other complex programs. Treating data objects and program objects equally helps to finely optimize the overlap of computation and data transfer on heterogeneous hardware platforms.

[0036] (2) Dependency directed cyclic graph.

[0037] An operator has input or output objects, and data has write or read operators. These relationships can be abstracted as dependencies between objects, represented by the edges of a directed graph, with objects as the vertices of the graph.

[0038] Unlike the directed acyclic graphs used by other systems, directed cyclic graphs can directly describe loops, coroutines and other control flows. Operator objects can also be used as input or output for other operators, which can implement functional programming models and unstructured control flows. Operators can be assembled from other operators, i.e. nested subgraphs, which can implement function calls and recursive control flows. Operators can be pure functions, have external side effects, be closures with internal states, or be auxiliary components that cooperate with the driver.

[0039] Therefore, such directed cyclic graphs can be used to describe complex and complete large programs.

[0040] (3) Script language design

[0041] DDQ uses a self-designed script language to describe directed cyclic graphs. This language is a purely functional, dynamically typed language. Unlike most programming languages, DDQ scripts are descriptive, and the order of statements is not important, so the running flow can be left to the scheduler to decide. DDQ scripts are easy to read and easy to write programs; at the same time, the abstract model is simple and flexible, suitable for other program analysis and optimization.

[0042] The main syntax element of a script is a script node, and the main syntax structure is the description of various relationships between script nodes. Among them, the binding relationship includes: binding a name for a script node, binding a built-in object for a script node, reading an object, an object list, etc.; the dependency relationship includes: the input and output relationship of operator objects and data objects, etc.; the equivalence relationship includes: the same script node, the same binding object, etc.; the hierarchical relationship includes: script node duplication, attribute relationship, etc.; the operation relationship includes: list pattern matching, built-in function function, subgraph calling, etc.

[0043] Among the objects that can be read into a script, there is a class saved in the file system, specified by multiple path layers and file names, called an operator library.

[0044] (4) Directed and cyclic graph state machine

[0045] The script describes a static directed and cyclic graph, which is stored as a dynamic ring list or ring queue by the scheduler at runtime, consisting of several nodes, each of which specifies an operator object in the directed and cyclic graph, input and output objects, and an execution driver. Each object defines its current state information.

[0046] The scheduler will query the state of the object in turn at runtime, and assign the control flow to the driver when the object state meets the specified conditions. For example, when the input and output of the operator are ready, the node will be scheduled to the specified driver for execution. The driver can return control after the operator starts running or completes running, and then the scheduler will continue to concurrently process other operators. When these action events occur, the object state is converted according to the specified rules.

[0047] The scheduler is responsible for managing memory and will automatically destruct nodes that are no longer needed. When all nodes are destructed, the program execution terminates.

[0048] (5) Software and hardware driver implementation

[0049] The driver of DDQ refers to the program interface responsible for running operators and managing data. DDQ is cross-hardware oriented, and accessing each hardware platform requires implementing the corresponding driver; similarly, operators developed based on different software environments or programming models sometimes also require specific drivers. For directed and cyclic graphs, scripts, schedulers, and most operators, the driver is transparent.

[0050] To support the development and implementation of various drivers, DDQ provides a set of independently designed stackless coroutine interfaces based on C language. With the support of this coroutine, the scheduler can schedule multiple same or different drivers simultaneously and automatically execute operators concurrently.

[0051] DDQ provides underlying support for the porting and optimization of programs, especially scientific application programs, on various runtime environments or hardware platforms. Through DDQ, programmers can only consider the abstraction and combination of operators, without considering the execution order and processor utilization on the runtime environment or hardware, which can adapt to different runtime environments and hardware, and reduce the development and debugging optimization time of personnel. Specifically, DDQ has the following technical advantages and effects:

[0052] (1) Adapt to various runtime environments and hardware platforms.

[0053] DDQ is a kind of operator scheduling framework that is common to various hardware, which can adapt to various heterogeneous and homogeneous hardware, especially new domestic hardware such as Shenwei processor and Maicun processor.

[0054] (2) Automatic implementation of operator scheduling.

[0055] DDQ can automatically schedule and execute operators on the hardware platform according to the directed cyclic graph specified by the script, reducing the development and debugging optimization time of personnel.

[0056] (3) Improve the utilization rate of hardware components.

[0057] DDQ will only be executed when the operator is ready according to the directed cyclic graph, otherwise it will traverse other operators. The execution order between operators on a single processor core is concurrent, which improves the utilization rate of hardware components.

[0058] (4) Facilitate the use of programmers and the development of operator library.

[0059] The operator and the directed cyclic graph are described by using a user-friendly script language. By using the script language, programmers can use DDQ very conveniently. Since the script function is independent of the order of instructions, the programming flexibility is improved, the user can express the logic of the computation graph in a more intuitive way, and the code readability and maintainability are improved. At the same time, since the script language calls the operator through the file path, the development of the operator library is relatively independent, and different operator libraries will not conflict with each other, thereby facilitating the development of the operator library. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 It is a schematic diagram of the overall framework of DDQ.

[0061] Figure 2 It is a schematic diagram of the composition of nodes in the data flow graph.

[0062] Figure 3 It is a schematic diagram of the Fibonacci sequence calculation represented by the directed cyclic graph of the operator. DETAILED DESCRIPTION

[0063] The present invention will be described in further detail below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0064] DDQ has the characteristics of performance portability, code reusability, concurrent and parallel execution of operators, data flow-based and directed cyclic graph program abstraction, and mainly includes program and data abstraction, dependency relationship directed cyclic graph, scripting language design, directed cyclic graph state machine, and software and hardware driver implementation.

[0065] Figure 1 This diagram shows the overall DDQ framework. From top to bottom, DDQ consists of a user interface, a middle layer, an operator library, and a scheduler. The user interface is intended for users or upper-layer frameworks. At this layer, operators in the operator library are called, hardware information for the operator execution is specified, and the information is assembled into a directed cyclic graph. The middle layer receives the directed cyclic graph from the upper layer, converts it into a data flow graph, and passes it to the lower-layer scheduler. The scheduler populates the nodes in the data flow graph with runtime information, traverses the nodes in the execution graph, and ultimately outputs the results.

[0066] The DDQ scheduler implementation primarily consists of three entities: operators, memory objects, and nodes. An operator is a structure containing a computation object in memory and its associated input and output objects. The most basic computation object is a function with a return value and two void** type parameters. More complex computation objects can be nested directed cyclic graphs or other forms that can be run by a specified driver. A node is a collection of operator mutable states that tracks operator execution status, data read and write progress, and carries relevant driver information. An operator's return value may include, but is not limited to, the following: task completion and operator retention, task completion and operator destruction, temporary transfer of control due to incomplete tasks, and task errors. The scheduler executes the next step based on the operator's return value and updates the status of related objects at specified stages. In this framework, operators are abstracted from computations within the program by developers or implemented by other upper-level frameworks.

[0067] Memory objects encapsulate all programs and data, including but not strictly distinguishing between compute objects and data objects. Memory objects can contain multiple mutually exclusive states, including pending allocation, pending construction, pending destruction, pending reading, reading, pending writing, writing, and packaged. Some objects can contain more complex state information. DDQ uses the state of an operator's input and output objects in a node to determine whether an operator can be executed. An operator is scheduled for execution only if the operator's states match, for example, if all input objects are readable and all output objects are writable.

[0068] like Figure 2As shown, a node is the basic unit of scheduling in the DDQ framework. DDQ groups nodes into a circular linked list or a circular queue, and traverses the nodes in sequence to check the state and execute the operators in the ready nodes during execution. Figure 1 In the middle scheduler, the right side shows an example of scheduling mode of concurrent execution of operators. The scheduler first executes the operator of node 1, gets the return value: the task is not completed, temporarily hands over the control, and then saves the intermediate result and releases the computing resources, and then traverses and executes node 2 and other nodes, and finally the scheduler executes the unexecuted part of the operator of node 1 and gets the return value: the task is completed and the destructor is executed, thereby realizing the concurrent execution of node 1 and the remaining nodes.

[0069] DDQ provides a scripting language for describing operators and directed cyclic graphs. The scripting language is read-write friendly and instruction order independent, and has high flexibility and readability. The scripts of the operators and the directed cyclic graphs are as follows.

[0070] The script of the addition operator:

[0071]

[0072] The script of the Fibonacci sequence directed cyclic graph:

[0073]

[0074] The process of describing operators and directed cyclic graphs by the scripting language mainly includes: calling of the operator, specification of input and output data, specification of data type, and specification of attribute. When describing the directed cyclic graph, the DDQ scripting language calls the operator through the load_op primitive and the file path of the operator script, and when describing the computing object, the DDQ scripting language calls the function implemented in C language through load_so. The DDQ script specifies the input and output data of the operator through “<” and “[]”. “<” represents a directed edge in the directed graph, and the left side is the starting point of the edge and the right side is the end point of the edge. The data is contained by “[]”, and the DDQ script uses variable names or variable types to specify data variables, supports integer, floating point number and string constants, and list and other combination types. The type of the variable is called through the load_type primitive and the file path of the variable type, and when the operator is called, the data type matching check is performed through the context of the directed cyclic graph script and the operator script. The DDQ script uses “=” to represent the equal relationship, and unlike the conventional programming language, the data on the left and right sides of “=” is equivalent, and has reflexivity, symmetry and transitivity.

[0075] Taking the calculation and printing of the Fibonacci sequence as an example, the specific implementation of each process of the framework is introduced in detail as follows.

[0076] (1) Program and data abstraction

[0077] To realize the operatorized scheduling of a program, first the different operations of the program need to be abstracted into operators with a uniform interface. An operator is a computational unit that contains input, output and computation. In the example of the present invention, as shown in Figure 3 , five kinds of operators are used, which are initialization operator, termination judging operator, copy operator, addition operator and print operator. The initialization operator initializes the input value into the output. The termination judging operator represents the termination condition judging of a loop, and adds 1 to the output variable each time until the output variable is greater than or equal to the input. The copy operator copies the value of the input variable into the output variable. The addition operator has two inputs and one output, and its output is equal to the sum of the input variables. The print operator has one input but no output, and its computation is to print the input to the standard output.

[0078] Figure 3 The circle node in the DDQ is a data object, which can be a constant or a variable, and can be of any simple or complex type. The DDQ manages it using its memory entry, but the internal information of it is transparent to the DDQ. The type derivation and other work have been pre-solved by the script compiler.

[0079] (2) Dependence directed acyclic graphing

[0080] The driving of the execution of the operators is specified, and the operators and the input and output data are organized in a directed acyclic graph, so as to express the complete program. The graph contains the dependence between the operators and the data and the correspondence between the operators and the back end. As shown in Figure 3 , the directed acyclic graph of the complete Fibonacci sequence program is composed of eight operators and seven data objects in this example, the square node represents the operator, and the circle node represents the data object. The execution driving of the operators in this example is "pthread", i.e. multi-thread parallel.

[0081] (3) Script language design

[0082] The DDQ provides a script language for describing the operators and the directed acyclic graph. The first line in the script of the addition operator indicates the position where the addition operator described by the script is implemented. The second line indicates that the execution back end of the operator is "direct", i.e. the operator is executed in a single-threaded manner. The third line describes the types and quantities of the input and output of the operator, and the addition operator has two real type inputs and one real type output.

[0083] In the operator of the Fibonacci sequence, the first row indicates that the addition operator has two inputs and one output, the input of the copy operator is the output of the addition operator, and the output of the copy operator is the input of the print operator, the second row is similar to the first row. The input of the initialization operator in the third row and the fourth row is an integer constant "1", and the output is a variable "a" and a variable "b", respectively, and they are used as the input of the addition operator in the first row.

[0084] (4) Directed acyclic graph state machine

[0085] The directed acyclic graph described by the script is a static abstract representation, and the script compiler disassembles all units that should be run into operators, and then packs the operators and all object information into the dispatcher. The dispatcher assigns the operator to the driver at runtime, and packs the state information of the operator as a node as a basic unit of scheduling. These nodes are stored in the form of a circular linked list or a circular queue in memory, and one node corresponds to one operator. In the example of Figure 3 , the circular linked list contains a total of eight nodes. The input and output of the operator in each node are packaged into a data object, which is a structure in C language, and in addition to the pointer of the data, it also contains some runtime state information. The initial state of the eight operators and three constant objects is "to be read", and the initial state of the four intermediate variable objects is "to be allocated". Each node can also contain other runtime states, such as the parity count of the write action. These nodes are traversed by the dispatcher and executed concurrently based on the state, and the state evolves until all nodes are destructed.

[0086] (5) Software and hardware driver implementation

[0087] DDQ is oriented to cross hardware and runtime environment, and access to various software and hardware platforms requires implementation of corresponding driver implementation, that is, the execution process of the operator on the software environment or hardware platform. The driver implementation is transparent to most operators. In this example, the driver used is "pthread", that is, multi-threaded execution mode. As Figure 2 shown, the present application also supports more hardware platforms and runtime environments, such as "cuda" indicating that the CUDA model of GPU is used to execute the calculation, "tianhe" indicating that the accelerator of the Meiya processor is used to execute the calculation, "direct" indicating that the single-threaded execution is used to execute the calculation, "fork" indicating that the multi-process mode is used to execute the calculation, and "sunway" indicating that the Sunway processor is used to execute the calculation.

[0088] While specific embodiments of the application have been disclosed in order to illustrate the application and to assist those skilled in the art in practicing the application, it is to be understood that various substitutions, modifications and changes can be made by those skilled in the art without departing from the spirit of the application and the scope of the appended claims. Accordingly, it is intended that the application not be limited, except by the scope of the claims.

Claims

1. A method for concurrently scheduling operators across hardware, comprising the following steps: 1) Abstracting the different operations of a target program into operators with a unified interface, and assigning a hardware and software platform for executing each operator; Abstract operators and their input and output data into objects respectively; 2) Determining dependencies between objects based on the target program's data flow, organizing the objects in a directed cyclic graph to obtain a directed cyclic graph as an abstract representation of the target program; the directed cyclic graph includes dependencies between operators and data, as well as correspondences between operators and software and hardware platforms; and providing a driver on each of the software and hardware platforms for executing the operators; 3) storing nodes corresponding to each operator according to the directed cyclic graph, where each node includes an operator and its input object, output object, and the software and hardware platform corresponding to the operator; 4) The scheduler traverses the stored nodes and obtains the currently executable node; Then, the coroutine technology is used to concurrently schedule the operators in the currently executable nodes to the corresponding hardware for execution, and receive the return values ​​of the operators.

2. The method according to claim 1, characterized in that A circular linked list, a circular queue or a state machine model is used to store the nodes corresponding to each operator.

3. The method according to claim 2, characterized in that The scheduler determines whether the corresponding node can be executed based on the operator's own status, input object status, and output object status in the node; when the operator itself and the input object are readable, the output object is writable, or when the data object of the operator is partially written, the node corresponding to the operator is an executable node.

4. The method according to claim 1, wherein The states of the object include: to be allocated, to be constructed, to be destroyed, to be read, reading, to be written, writing, and packaged.

5. The method according to claim 1, wherein The hardware driver is an abstract interface that has nothing to do with hardware.

6. The method according to claim 1, characterized in that In step 1), the operator and the directed cyclic graph are described using a scripting language; wherein, the call, input and output data, data type and attributes of the operator are specified using the scripting language; wherein, the input and output data of the operator are specified using "<" and "[]", "<" represents a directed edge in the directed cyclic graph, the data is contained in "[]" and is separated by spaces, and the data variable is specified using the variable name or variable type; the type of the variable is called using the load_type primitive and the file path of the variable type, and when calling the operator, a data type matching check is performed through the context of the directed cyclic graph script and the operator script; "=" is used to indicate an equal relationship, and the data on the left and right sides of "=" are equivalent, with reflexivity, symmetry and transitivity.

7. The method according to claim 6, characterized in that The grammatical elements of the script in the script language are set as script nodes, and the grammatical structure is a description of various relationships between script nodes, including binding relationships, dependency relationships, equality relationships, hierarchical relationships and operation relationships; among which, the binding relationships include: binding names to script nodes, binding built-in objects to script nodes, reading objects, and object lists; the dependency relationships include: input and output relationships between operator objects and data objects; the equality relationships include: the same script nodes and the same binding objects; the hierarchical relationships include: script node replication and attribute relationships; the operation relationships include: pattern matching of lists, built-in function functions, and subgraph calls.

8. The method according to claim 1, characterized in that The operator is a function with a return value and two void** type parameters; the return values ​​include: task completed and operator retained, task completed and operator destroyed, task not completed and control temporarily transferred, task error; when the return value of the operator is task completed, the scheduler obtains the node to be destroyed from the circular linked list and deconstructs it.

9. A cross-hardware operator concurrent scheduling system, characterized in that: Includes operator library, operation abstraction unit, operator directed graph unit, ring linked list generation unit and scheduler; The operation abstraction unit is used to abstract different operations of a target program into operators with a unified interface, and to specify a software and hardware platform for executing each operator; Abstract operators and their input and output data into objects respectively; The operator library is used to store the operators; The operator directed graph unit is used to determine the dependency relationships between objects based on the data flow of the target program, organize the objects in a directed cyclic graph, and obtain a directed cyclic graph as an abstract representation of the target program; the directed cyclic graph contains the dependency relationships between operators and data, as well as the correspondence between operators and software and hardware platforms; a driver is provided on each of the software and hardware platforms for executing the operators; The ring linked list generating unit is used to store nodes corresponding to each operator according to the directed cyclic graph, each node including an operator and its input object, output object and the software and hardware platform corresponding to the operator; The scheduler is used to traverse the stored nodes and obtain the currently executable nodes; Then, the coroutine technology is used to concurrently schedule the operators in the currently executable nodes to the corresponding hardware for execution, and receive the return values ​​of the operators.

10. The system according to claim 9, characterized in that The scheduler determines whether the corresponding node can be executed based on the operator's own status, input object status, and output object status in the node; when the operator itself and the input object are readable, the output object is writable, or when the data object of the operator is partially written, the node corresponding to the operator is an executable node.

Images