Method and apparatus for heterogeneous parallel computing

Abstract

The present invention provides a heterogeneous parallel computing method and apparatus. Among them, the method includes distributing each operator to a corresponding heterogeneous engine group according to different computing power requirements of the operators; numbering the operators distributed to each heterogeneous engine group; parsing the data dependency relationships between the operators in each heterogeneous engine group, and inserting a counter comparison command word into the operator queue, wherein the data dependency relationships are recorded by the numbers of the operators in each heterogeneous engine; performing in-group synchronization on each sub-engine in the same heterogeneous engine group; and performing inter-group synchronization on each heterogeneous engine group. The technical solution provided by the present invention can reduce the overhead of synchronization events while making full use of the computing power of GPUs and AI accelerators, thereby significantly improving the utilization rate of hardware resources and enhancing the overall performance of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of parallel computing, and in particular, to a method and device for heterogeneous parallel computing. Background Art

[0002] In a heterogeneous computing system, the collaborative computing between a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU), or an Artificial Intelligence (AI) computing accelerator is a key technology for achieving efficient parallel processing. The CPU is usually responsible for the control flow and the execution of serial tasks, and dispatches compute-intensive operators to the GPU or AI accelerator for computing, and obtains the computing results by creating synchronization events. In this process, the on-chip general controller is responsible for docking the tasks dispatched by the CPU, scheduling hardware resources, and responding to event synchronization operations. The latency overhead of event synchronization control is the main factor affecting system performance, and its latency range may vary from dozens of microseconds to thousands of microseconds.

[0003] [[ID=ll]]Specifically, the latency mainly comes from the following aspects:

[0004] 1. Command submission overhead: The communication latency generated when the CPU dispatches tasks to the GPU or AI accelerator;

[0005] 2. Hardware scheduling latency: The task scheduling time inside the GPU or AI accelerator;

[0006] 3. Event signal transmission latency: The synchronization overhead between computing engines inside the GPU or AI accelerator;

[0007] 4. Latency for the CPU to obtain the event status: Including the time for query or interrupt response.

[0008] In recent years, with the diversified development of AI models, in order to meet the requirements of different computing tasks, the GPU and AI accelerator generally adopt a heterogeneous computing power architecture, such as integrating various computing resources such as tensor computing engines, vector computing engines, vector computing engines, and communication engines. These resources are usually uniformly scheduled by the on-chip general controller to improve computing efficiency.

[0009] In the programming models of the GPU and AI accelerator, a task queue (Stream) is the basic unit of task scheduling. The operators within the same Stream are strictly executed in order, while the operators of different Streams can be asynchronously and parallelly computed. For example, in a simple AI model containing three operators (A, B, C), if there is no data dependency between operator A and operator B, while operator C depends on the output results of operator A and operator B, the traditional scheduling method has the following problems:

[0010] 1. Single Stream Scheduling: If all operators are executed in the same stream, the total controller of the GPU or AI accelerator needs to schedule operators A, B, and C sequentially in order, resulting in a total latency that is the sum of the execution times of the three operators and the time of one synchronization event (the startup and execution time of operator A + the startup and execution time of operator B + the startup and execution time of operator C + the time of one CPU synchronization event). This mode cannot fully utilize the parallelism of the hardware computing power, especially when the GPU or AI accelerator has the ability to execute operators A and B in parallel, and its performance bottleneck is more significant.

[0011] 2. Multi-Stream Scheduling: By allocating operators A and B to different streams to improve parallelism and resolving data dependencies by adding event synchronization on the CPU side, the total latency can be optimized to the execution time of the slower one of operators A and B plus the execution time of operator C and the synchronization overhead [max (the startup and execution time of operator A + the time of one CPU synchronization event, the startup and execution time of operator B + the time of one CPU synchronization event) + the startup and execution time of operator C + the time of one CPU synchronization event]. However, although this method increases the parallelism of the on-chip computing power resources, it increases the CPU synchronization event time. In particular, with the increase in the complexity of the AI model, frequent event synchronization will lead to additional latency accumulation, which may instead become a key factor restricting performance.

[0012] Therefore, in the prior art, how to improve the parallelism of heterogeneous computing power while reducing the overhead of synchronization events has become a technical problem to be solved urgently. Summary of the Invention

[0013] In view of this, the present invention provides a method and device for heterogeneous parallel computing to solve the above technical problems in the prior art.

[0014] According to one aspect of the present invention, a method for heterogeneous parallel computing is provided, wherein the method includes:

[0015] Distribute each operator to the corresponding heterogeneous engine group according to the different computing power requirements of the operators, and each heterogeneous engine group includes multiple sub-engines for executing operators;

[0016] Number the operators distributed to each heterogeneous engine group;

[0017] Analyze the data dependency relationships between the operators in each heterogeneous engine group and insert counter comparison command words into the operator queue;

[0018] Perform intra-group synchronization on the sub-engines in the same heterogeneous engine group, and the intra-group synchronization includes:

[0019] Each sub - engine generates its respective sub - engine corresponding group completion flag after completing the operator calculation;

[0020] Receive all sub - engine corresponding group completion flags to achieve in - group synchronization;

[0021] Perform inter - group synchronization for each heterogeneous engine group. The inter - group synchronization includes:

[0022] Each heterogeneous engine group generates its respective operator completion pulse signal after completing the operator calculation;

[0023] Receive the operator completion pulse signals of each heterogeneous engine group and use a counter for counting. The counter value is the sequence number of the operators that have been processed and completed by each heterogeneous engine group;

[0024] Each heterogeneous engine group obtains the configured threshold of the counter according to the data dependency relationship;

[0025] Compare the counter value with the configured threshold. When the counter value is greater than or equal to the configured threshold, an enabling signal for the heterogeneous engine group is issued to achieve inter - group synchronization.

[0026] According to another aspect of the present invention, there is provided a device for heterogeneous parallel computing. Wherein, the device includes a compiler and heterogeneous engine groups,

[0027] The compiler includes:

[0028] An operator attribution distribution module, configured to distribute each operator to the corresponding heterogeneous engine group according to the different computing power requirements of the operators. Each heterogeneous engine group includes multiple sub - engines for executing operators;

[0029] An operator sorting module, configured to number the operators distributed to each heterogeneous engine group;

[0030] An operator - to - operator dependency analysis module, configured to analyze the data dependency relationship between each operator in each heterogeneous engine group and insert a counter comparison command word into the operator queue;

[0031] The heterogeneous engine group includes:

[0032] An in - group hardware synchronization sub - module, configured to perform in - group synchronization for each sub - engine in the same heterogeneous engine group. The in - group synchronization includes:

[0033] Each sub - engine generates its respective sub - engine corresponding group completion flag after completing the operator calculation;

[0034] Receive all sub - engine corresponding group completion flags to achieve in - group synchronization;

[0035] An inter - group hardware synchronization sub - module, configured to perform inter - group synchronization for each heterogeneous engine group. The inter - group synchronization includes:

[0036] Each heterogeneous engine group generates its own operator completion pulse signal after completing the operator calculation;

[0037] Receive the operator completion pulse signals of each heterogeneous engine group and use a counter to count. The counter value is the sequence number of the operators that have been processed and completed by each heterogeneous engine group;

[0038] Each heterogeneous engine group obtains the configuration threshold of the counter according to the data dependency relationship;

[0039] Compare the counter value with the configuration threshold. When the counter value is greater than or equal to the configuration threshold, an enabling signal for the heterogeneous engine group is issued to achieve inter-group synchronization.

[0040] As can be seen from the above technical solutions, the technical solution provided by the present invention can significantly improve the performance of the heterogeneous computing system through an innovative on-chip synchronization mechanism, and it has at least the following advantages:

[0041] 1. Through the on-chip synchronization mechanism, operators without data dependency relationships can be identified and their parallel computing can be improved on the GPU or AI accelerator, thereby significantly improving the utilization rate of hardware resources and accelerating the task execution efficiency;

[0042] 2. By adopting an optimized on-chip synchronization circuit design, the traditional event synchronization overhead is reduced from dozens of microseconds to dozens of nanoseconds, greatly reducing the synchronization waiting time and improving the overall computing throughput;

[0043] 3. By introducing multiple sets of synchronization mechanisms, the system can support multiple processes to simultaneously call the on-chip hardware computing power resources, achieve more efficient resource reuse, and improve the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings are used to provide a further understanding of the technical solutions of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, but do not constitute a limitation to the technical solutions of the present invention.

[0045] Figure 1 Shows an AI model structure including three operators;

[0046] Figure 2 Shows a method of executing three operators using the same task queue;

[0047] Figure 3 Shows a method of executing three operators using multiple task queues;

[0048] Figure 4 Shows a schematic diagram of the hardware resources adopted in the exemplary embodiment of the present invention;

[0049] Figure 5 Shows a schematic diagram of compiler-assisted operator dispatch in the method provided by an exemplary embodiment of the present invention;

[0050] Figure 6 Shows a schematic diagram of in-group synchronization within a single engine group in the method provided by an exemplary embodiment of the present invention;

[0051] Figure 7 Shows a schematic diagram of inter-group synchronization between multiple engine groups in the method provided by an exemplary embodiment of the present invention;

[0052] Figure 8 Shows a schematic diagram of software resources adopted in an exemplary embodiment of the present invention;

[0053] Figure 9 Shows a structural block diagram of the device provided by an exemplary embodiment of the present invention. Detailed implementation manners

[0054] Next, various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and does not impose any limitation on the present invention and its application or use. The present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present invention thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0055] Unless explicitly stated, if the number of elements is not specifically limited, the element can be one or more. The term "multiple / several" means two or more, the term "based on" should be interpreted as "at least partially based on", the term "and / or" and "at least one of..." cover any one of the listed items and all possible combinations.

[0056] Please refer to Figures 1 to 3 , which shows the synchronous operator calculation method adopted in the background art.

[0057] Figure 1 Shows an AI model structure including three operators, where there is no data dependency between operator A and operator B, and the execution of operator C depends on the outputs of operator A and operator B.

[0058] Figure 2 Shows a method of executing three operators using the same task queue (Stream). The CPU creates operators A, B, and C of the AI model in the same task queue and dispatches them to the GPU or AI accelerator for execution. The operators within the same task queue are executed strictly in order, and the scheduling process of the on-chip general controller is as follows:

[0059] 1. Execute Operator A: The CPU dispatches Operator A to the GPU or AI accelerator, starts the calculation, and waits for completion. The time consumed is denoted as T A (including the startup latency and execution time of Operator A).

[0060] 2. Execute Operator B: After Operator A is completed, the GPU or AI accelerator starts to execute Operator B. The time consumed is denoted as T B (including the startup latency and execution time of Operator B).

[0061] 3. Execute Operator C: Since Operator C depends on the outputs of Operator A and Operator B, it needs to wait for the completion of the former two before starting to execute. The time consumed is denoted as T C (including the startup latency and execution time of Operator C).

[0062] 4. CPU Synchronization Event: The CPU obtains the final result through polling or interrupt. The synchronization overhead is denoted as T sync .

[0063] It can be seen that the total time T1 = T A + T B + T C + T sync . In this scheduling and execution method, although the synchronization overhead is low, the parallelism of the computing power cannot be fully utilized, and the computing power utilization rate is low.

[0064] Figure 3 Shows a method of executing three operators using multiple task queues. The CPU dispatches Operator A and Operator B to different Streams (Stream0 and Stream1) respectively to make full use of the parallel computing power. However, Operator C still depends on the outputs of both, and needs to wait for the synchronization to complete before execution. The scheduling process of the on-chip total controller is as follows:

[0065] 1. Execute Operators A and B in parallel:

[0066] The CPU dispatches Operator A to Stream0, and the time consumed is the sum of T A and the synchronization overhead T syncA .

[0067] The CPU dispatches Operator B to Stream1, and the time consumed is the sum of T B and the synchronization overhead T syncB .

[0068] 2. Synchronization Wait:

[0069] The CPU needs to wait for both Operator A and Operator B to complete. Therefore, the time consumed is max(T A + T syncA , T B + T syncB ).

[0070] 3. Execute operator C alone:

[0071] After both operator A and operator B are completed, the CPU dispatches operator C to Stream2, which takes time T C And the synchronization overhead T syncC The sum.

[0072] It can be seen from this that the total time T2 = max(T A + T syncA , T B + T syncB ) + T C + T sync . This scheduling and execution method will result in an increase in additional CPU synchronization events (T syncA , T syncB , T sync ). In complex AI models, frequent synchronization may lead to cumulative delays and affect system performance.

[0073] In view of this, the present invention proposes a heterogeneous parallel computing method and device, which can further reduce synchronization delay while improving system parallelism and optimize the overall performance of the system.

[0074] Please refer to Figure 4 , which shows a schematic diagram of the hardware resources adopted in the exemplary embodiment of the present invention.

[0075] Figure 4 In

[0076] , three groups of heterogeneous engine groups A[m], B[n], and C[k] are taken as examples for illustration, but it does not limit the present invention. In actual implementation, the number of heterogeneous engine groups is not limited to three groups.

[0077] Please refer to Figure 5 , which shows a schematic diagram of operator dispatch in the method provided by the exemplary embodiment of the present invention.

[0078] The present invention provides a heterogeneous parallel computing method. Specifically, the method provided by the present invention includes:

[0079] According to the different computing power requirements of operators, each operator is distributed to the corresponding heterogeneous engine group, and each heterogeneous engine group includes multiple sub-engines for executing operators;

[0080] Number the operators distributed to each heterogeneous engine group;

[0081] Analyze the data dependency relationships among the operators in each heterogeneous engine group, and insert counter comparison command words into the operator queue, where the data dependency relationships are recorded by the numbers of the operators in each heterogeneous engine;

[0082] Perform in-group synchronization for the sub-engines in the same heterogeneous engine group;

[0083] Perform inter-group synchronization for the heterogeneous engine groups.

[0084] Please also refer to Figure 6 , which shows a schematic diagram of in-group synchronization within a single engine group in the method provided by the exemplary embodiments of the present invention. Among them, the in-group synchronization includes:

[0085] Each sub-engine generates its own sub-engine corresponding group completion flag after completing the operator calculation;

[0086] Receive all the sub-engine corresponding group completion flags to achieve in-group synchronization.

[0087] It should be noted that Figure 6 is only exemplary, which schematically shows a set of in-group synchronization resources. However, in actual implementation, multiple sets of in-group synchronization resources can be introduced and the completion status of different operators can be distinguished by using the numbers (IDs) of the operators, so as to fully utilize the computing resources and achieve the parallelism of multiple homogeneous operators in the same task queue.

[0088] Please also refer to Figure 7 , which shows a schematic diagram of inter-group synchronization among multiple engine groups in the method provided by the exemplary embodiments of the present invention. Among them, the inter-group synchronization includes:

[0089] Each heterogeneous engine group generates its own operator completion pulse signal after completing the operator calculation;

[0090] Receive the operator completion pulse signals of each heterogeneous engine group and use a counter to count. The counter value of the counter is the serial number of the operators that have been processed and completed by each heterogeneous engine group;

[0091] It should be noted that Figure 7 is only exemplary, which schematically shows a set of inter-group synchronization resources. However, in actual implementation, multiple sets of inter-group synchronization resources can be introduced and the execution status of the operators in different task queues can be distinguished by using the numbers (stream IDs) of the task queues. At this time, the counter threshold comparison part needs to select the correct counter value for comparison according to the number of the task queue.

[0092] Each heterogeneous engine group obtains the configuration threshold of the counter according to the data dependency relationship;

[0093] Compare the counter value with the configured threshold. When the counter value is greater than or equal to the configured threshold, an enabling signal for the heterogeneous engine group is issued to achieve fast event synchronization at the cycle level.

[0094] Execute the three operators A, B, and C in the technical background using the method provided by the technical solution of the present invention, and the total time consumption is shortened to max(T A , T B ) + T C +T sync . This fully utilizes the computing power resources on the GPU and AI accelerator chips and does not introduce CPU event synchronization time. When the computing scale is more complex and larger, its performance benefits will be more prominent.

[0095] In a preferred embodiment, the inter-group synchronization further includes masking the sub-engines that do not participate in the execution in each heterogeneous engine group to further save the system execution time.

[0096] In a preferred embodiment, the inter-group synchronization further includes when all the sub-engines participating in the calculation in a certain heterogeneous engine group complete the calculation, broadcasting the corresponding operator completion pulse signal to all other heterogeneous engine groups.

[0097] In addition, those skilled in the art know that software acceleration can also be used, that is, a controller and an operator status counter are provided for each heterogeneous engine group, and then multiple atomic counter managements are set in the memory space visible to all heterogeneous engines in combination with software and controller firmware.

[0098] In the software acceleration scheme, each counter operation is on the order of dozens of cycles, but the total overhead increases linearly with the number of counter operations, which will make the synchronization delay and overhead 100 times larger than the hardware acceleration scheme, and the order of thousands of cycles to tens of thousands of cycles varies depending on the busyness of the memory space.

[0099] Please refer to Figure 8 , which shows a schematic diagram of the software resources used in the exemplary embodiment of the present invention.

[0100] Still taking three heterogeneous engine groups A[m], B[n], and C[k] as an example for illustration. In this exemplary resource architecture, each of the heterogeneous engine groups A[m], B[n], and C[k] is equipped with a controller CTRL, an operator status counter Status, and multiple groups of atomic counter resources, and it uses controller firmware and a software compiler to implement operator synchronization and data dependency management.

[0101] The following takes the heterogeneous engine group A[m] as an example to exemplarily illustrate the intra-group synchronization of the heterogeneous engine group. The intra-group synchronization operations of other heterogeneous engine groups are the same as this, so they will not be described again.

[0102] When the compiler generates the operator task list, it needs to insert an atomic operation on the shared memory after each operator. This operation can be an atomic counter or a similar method to indicate that all sub-engines participating in the calculation of the current operator have completed the corresponding calculation. If all sub-engines A[0], A[1], …… A[m-1] in the heterogeneous engine group A[m] participate in the calculation of the current operator, the threshold of the atomic counter is m. If only x sub-engines participate in the calculation, the threshold of the atomic counter is x. When multiple operators are parallel, multiple groups of atomic counters need to be provided to ensure the correct operation of the system function.

[0103] In the case where x sub-engines participate in the calculation, when all these x sub-engines complete the calculation tasks, the sub-engine task parser performs an atomic operation on the shared memory, and the intra-group synchronization overhead time is denoted as T_atomic×x.

[0104] The controller CTRL_A of the heterogeneous engine group A[m] determines the completion status of the current operator by monitoring the atomic counter corresponding to the operator.

[0105] The following describes the inter-group synchronization of each heterogeneous engine group.

[0106] When the controller CTRL of each heterogeneous engine group detects that an operator is completed, it updates its own completed operator status counter. It should be noted that the operator status counter of each heterogeneous engine group should be visible to the controllers of other heterogeneous engine groups.

[0107] The compiler expresses the inter-group data dependence relationship of the operators to the controllers of each heterogeneous engine group in the form of event management. Then, the controllers of each heterogeneous engine group check the operator status counters of each operator to resolve the data dependence relationship. The overhead of each check is on the order of dozens of cycles, and the total overhead will increase linearly with the number of checks.

[0108] Please refer to Figure 9 , which shows the structural block diagram of the device provided by the exemplary embodiment of the present invention.

[0109] The present invention also provides a device for heterogeneous parallel computing. Specifically, the device provided by the present invention includes a compiler and a heterogeneous engine group. Among them, the compiler includes an operator attribution dispatch module, an operator sorting module, and an inter-operator dependence relationship analysis module.

[0110] Specifically, the operator attribution and dispatch module is configured to distribute each operator to the corresponding heterogeneous engine group according to the different computing power requirements of the operators. Each heterogeneous engine group includes multiple sub-engines for executing the operators; the operator sorting module is configured to number the operators distributed to each heterogeneous engine group; the inter-operator dependency analysis module is configured to analyze the data dependencies between the operators in each heterogeneous engine group and insert a counter comparison command word into the operator queue.

[0111] The heterogeneous engine group includes an intra-group hardware synchronization sub-module and an inter-group hardware synchronization sub-module. Each heterogeneous engine group may also include a task parser.

[0112] Specifically, the intra-group hardware synchronization sub-module is configured to perform intra-group synchronization for the sub-engines in the same heterogeneous engine group; the inter-group hardware synchronization sub-module is configured to perform inter-group synchronization for the heterogeneous engine groups.

[0113] Among them, the intra-group synchronization performed by the intra-group hardware synchronization sub-module includes:

[0114] Each sub-engine generates its own sub-engine corresponding group completion flag after completing the operator calculation;

[0115] Receive all the sub-engine corresponding group completion flags to achieve intra-group synchronization;

[0116] Among them, the intra-group synchronization performed by the inter-group hardware synchronization sub-module includes:

[0117] Each heterogeneous engine group generates its own operator completion pulse signal after completing the operator calculation;

[0118] Receive the operator completion pulse signals of each heterogeneous engine group and use a counter to count. The counter value of the counter is the serial number of the operators that have been processed and completed by each heterogeneous engine group;

[0119] Each heterogeneous engine group obtains the configuration threshold of the counter according to the data dependency;

[0120] Compare the counter value with the configuration threshold. When the counter value is greater than or equal to the configuration threshold, an enabling signal for the heterogeneous engine group is issued to achieve inter-group synchronization.

[0121] The inter-group hardware synchronization sub-module is also configured to mask the sub-engines that do not participate in the execution in each heterogeneous engine group.

[0122] The inter-group hardware synchronization sub-module is also configured to broadcast its corresponding operator completion pulse signal to all other heterogeneous engine groups when all the sub-engines participating in the calculation in a certain heterogeneous engine group have completed the calculation.

[0123] Specifically, the compiler can implement the following functions:

[0124] Operator attribution and dispatch: Allocate operators to the corresponding heterogeneous engine groups according to computing power requirements (such as tensor / vector calculations);

[0125] Operator sorting and analysis of dependencies between operators: Generate serial numbers for each operator as the basis for comparing dependencies in the inter-group hardware synchronization module;

[0126] Dynamic computing power allocation: Apply for corresponding computing power resources according to the computing power requirements of operators. In the case of homogeneous computing power scenarios, allocate operators to the same heterogeneous engine group, and the in-group synchronization module manages the execution order; in heterogeneous computing power scenarios, allocate operators to different heterogeneous engine groups and rely on the inter-group synchronization circuit for coordination. Specifically, still taking the model structure in the background technology Figure 1 as an example, when operators A and B use homogeneous computing power, the compiler can distribute the homogeneous computing power within a single engine group to operators A and B according to the computing power requirements, and use the in-group hardware synchronization sub-module to record the completion status of single or multiple homogeneous operators; when operators A and B use heterogeneous computing power, the compiler distributes operators A and B to the corresponding heterogeneous engine groups, and each heterogeneous engine group uses the inter-group hardware synchronization sub-module to record the completion status of the operators.

[0127] It should be understood that Figure 9 the device shown in can correspond to the method described earlier in this specification. Thus, the operations, features, and advantages described above for the method also apply to the device provided by the present invention and the modules it includes, and the operations, features, and advantages described above for the device and the modules it includes also apply to the method provided by the present invention. For the sake of brevity, some operations, features, and advantages will not be repeated.

[0128] Although the specific functions have been discussed above with reference to specific modules, it should be noted that the functions of each module in the technical solution of the present invention can also be implemented by dividing them into multiple modules, and / or at least some functions of multiple modules can be combined into a single module for implementation. The manner in which a specific module in the technical solution of the present invention performs an action includes that the specific module itself performs the action, or the specific module calls or otherwise accesses an action (or performs the action in combination with the specific module). Therefore, the specific module that performs the action can include the specific module itself that performs the action and / or another module that the specific module calls or otherwise accesses and performs the action.

[0129] The technical solutions described in this invention are not limited to the specific examples of the described technical solutions. The explanations and descriptions of this invention in the foregoing text and drawings are not restrictive. For those skilled in the art, it is obvious that this invention is not limited to the details of the above exemplary embodiments, and it is also possible to implement this invention in other specific forms without departing from the spirit or basic characteristics of this invention. Therefore, the scope of protection required by this invention is defined by the claims rather than the above descriptions, and all changes falling within the meaning and scope of the equivalent elements of the claims are covered by the protection scope of this invention.

Claims

1. A heterogeneous parallel computing method, characterized in that, The method includes: Distributing each operator to a corresponding heterogeneous engine group according to the different computing power requirements of the operators, where each heterogeneous engine group includes multiple sub-engines for executing the operators; Numbering the operators distributed to each heterogeneous engine group; Analyzing the data dependency relationships between the operators in each heterogeneous engine group and inserting counter comparison command words into the operator queue; Performing intra-group synchronization on the sub-engines in the same heterogeneous engine group, and the intra-group synchronization includes: Each sub-engine generates its own sub-engine corresponding group completion flag after completing the operator calculation; Receiving all sub-engine corresponding group completion flags to achieve intra-group synchronization; Performing inter-group synchronization on the heterogeneous engine groups, and the inter-group synchronization includes: Each heterogeneous engine group generates its own operator completion pulse signal after completing the operator calculation; Receiving the operator completion pulse signals of each heterogeneous engine group and using a counter to count, where the counter value of the counter is the sequence number of the operators that have been processed and completed by each heterogeneous engine group; Each heterogeneous engine group obtains the configuration threshold of the counter according to the data dependency relationship; Comparing the counter value with the configuration threshold, and when the counter value is greater than or equal to the configuration threshold, an enabling signal for the heterogeneous engine group is issued to achieve inter-group synchronization; Wherein, the data dependency relationship refers to whether the calculation of an operator depends on the output of other operators.

2. The method according to claim 1, wherein The method further includes removing the on-chip general controller.

3. The method according to claim 1, characterized in that, The inter-group synchronization further includes masking the sub-engines in each heterogeneous engine group that do not participate in the execution.

4. The method according to claim 1, characterized in that The inter-group synchronization further includes when all the sub-engines participating in the calculation in a certain heterogeneous engine group have completed the calculation, broadcasting its corresponding operator completion pulse signal to all other heterogeneous engine groups.

5. The method according to claim 1, wherein The data dependency relationship is recorded by the numbers of the operators in each heterogeneous engine.

6. A heterogeneous parallel computing device, characterized in that, The device includes a compiler and heterogeneous engine groups, The compiler includes: An operator attribution dispatching module configured to distribute each operator to a corresponding heterogeneous engine group according to the different computing power requirements of the operators, where each heterogeneous engine group includes multiple sub-engines for executing the operators; An operator sorting module configured to number the operators distributed to each heterogeneous engine group; An inter-operator dependency relationship analysis module configured to analyze the data dependency relationships between the operators in each heterogeneous engine group and insert counter comparison command words into the operator queue; The heterogeneous engine group includes: An intra-group hardware synchronization sub-module configured to perform intra-group synchronization on the sub-engines in the same heterogeneous engine group, and the intra-group synchronization includes: Each sub-engine generates its own sub-engine corresponding group completion flag after completing the operator calculation; Receiving all sub-engine corresponding group completion flags to achieve intra-group synchronization; An inter-group hardware synchronization sub-module configured to perform inter-group synchronization on the heterogeneous engine groups, and the inter-group synchronization includes: Each heterogeneous engine group generates its own operator completion pulse signal after completing the operator calculation; Receiving the operator completion pulse signals of each heterogeneous engine group and using a counter to count, where the counter value of the counter is the sequence number of the operators that have been processed and completed by each heterogeneous engine group; Each heterogeneous engine group obtains the configuration threshold of the counter according to the data dependency relationship; Compare the counter value with the configured threshold. When the counter value is greater than or equal to the configured threshold, an enable signal for the heterogeneous engine group is issued to achieve inter-group synchronization; Among them, the data dependency refers to whether the calculation of an operator depends on the output of other operators.

7. The device according to claim 6, characterized in that, The device does not include an on-chip master controller.

8. The device according to claim 6, characterized in that Each heterogeneous engine group further includes a task parser.

9. The device according to claim 6, characterized in that, The inter-group hardware synchronization sub-module is further configured to mask the sub-engines that do not participate in the execution in each heterogeneous engine group.

10. The device according to claim 6, characterized in that, The inter-group hardware synchronization sub-module is further configured to broadcast its corresponding operator completion pulse signal to all other heterogeneous engine groups when all the sub-engines participating in the calculation in a certain heterogeneous engine group have completed the calculation.

11. The device according to claim 6, characterized in that, The data dependency is recorded by the numbers of the respective operators in each heterogeneous engine.

Images