Heterogeneous computing processing method and related device

Abstract

The invention discloses a heterogeneous computing processing method and a related device, and relates to the field of information processing.The heterogeneous computing processing method comprises the steps that a task completion signal sent by a first processor is received, and the task completion signal represents that the first processor completes a first computing subtask; obtaining a target scheduling code based on the task completion signal; generating a task starting signal according to the task completion signal and the target scheduling code; and based on the task starting signal, triggering a second processor to execute a second calculation sub-task according to a sub-task file, wherein the second calculation sub-task and the first calculation sub-task belong to the same calculation task.

Description

Technical Field

[0001] The present application relates to the field of information processing, and in particular to a heterogeneous computing processing method and related devices. Background Art

[0002] Heterogeneous computing is a distributed computing system that includes multiple computing units, such as CPU (central processing unit), NPU (Neural Processor Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), etc.

[0003] Figure 1 This is a schematic diagram of the existing heterogeneous computing architecture, including: CPU, NPU, DSP and GPU, and each processor is connected by a bus. Among them, the general computing unit CPU is responsible for computing scheduling, and the dedicated computing unit (NPU, GPU, DSP, etc.) is responsible for dense computing. Each XPU (a general term for multiple computing units) has different characteristics. The CPU is suitable for process control, the DSP is suitable for vector operations and has a certain degree of flexibility, the GPU is suitable for large-scale parallel computing, and the NPU is suitable for dedicated AI (Artificial Intelligence) computing.

[0004] Each XPU comes from a shelf product from a different supplier, and its computing power modules and capabilities are not fully configurable. For example, different XPUs support different types of data, and different operators are implemented and executed differently on different XPUs.

[0005] In order to support AI mixed-precision computing, in the existing heterogeneous computing architecture, all inferences are performed by XPUs that support certain types of data types or operators. For example, INT8+FP16 can be performed by GPUs for whole-network inference. The disadvantage is that the energy efficiency of GPU inference is lower than that of NPU inference, which leads to slower computing efficiency. Summary of the invention

[0006] The first aspect of the present application provides a heterogeneous computing processing method, including:

[0007] Receiving a task completion signal sent by the first processor, wherein the task completion signal indicates that the first processor has completed the first computing subtask;

[0008] Obtaining a target scheduling code based on the task completion signal;

[0009] Generate a task start signal according to the task completion signal and the target scheduling code;

[0010] The second processor is triggered based on the task start signal to execute a second computing subtask according to the subtask file, and the second computing subtask and the first computing subtask belong to the same computing task.

[0011] In a possible implementation, the generating a task start signal according to the task completion signal and the target scheduling code includes:

[0012] Parsing the target scheduling code to obtain a first part code and a second part code of the target scheduling code;

[0013] Determining, based on the first partial code and the task completion signal, that each pre-order processor completes each corresponding first computing subtask, the pre-order processor is a processor for executing a first computing subtask, the first computing subtask is a pre-order subtask of the second computing subtask, and the pre-order processor of the second computing subtask includes at least one first processor;

[0014] Determining an execution processor of a second computing subtask according to the second part of the code;

[0015] The execution processor corresponding to the second part of the code generates a task start signal, and the task start signal is used to trigger the execution processor to execute the second computing subtask.

[0016] In a possible implementation, after triggering the second processor to execute the second computing subtask according to the subtask file based on the task start signal, the method further includes:

[0017] Control the second processor to obtain the execution result of the first computing subtask according to the storage address corresponding to the subtask file in the preset storage area, so as to execute the second computing subtask according to the execution result of the first computing subtask, and the preset storage area is pre-allocated with a storage address for each computing subtask.

[0018] In a possible implementation, there are multiple first computing subtasks to be completed before the second computing subtask, and obtaining the target scheduling code based on the task completion signal includes:

[0019] Determining, based on the task completion signal, that the first first computing subtask corresponding to the second computing subtask is completed, and obtaining a target scheduling code in a local preset scheduling code queue in response to the task completion signal;

[0020] Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes:

[0021] In response to the task completion signal, determining the completion status of each first computing subtask;

[0022] Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

[0023] In a possible implementation, the method further includes:

[0024] Based on the idle state in the scheduling code queue, at least one scheduling code is read from the scheduling codes stored in the preset memory and put into the scheduling code queue, and the computing subtasks corresponding to the scheduling codes stored in the preset memory constitute a computing task.

[0025] In a possible implementation, obtaining a target scheduling code based on the task completion signal includes:

[0026] Determine that the first computing subtask is completed based on the task completion signal, and determine the target scheduling code corresponding to the second computing subtask that follows the first computing subtask according to a preset linked list sequence;

[0027] Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes:

[0028] In response to the task completion signal, determining the completion status of each first computing subtask;

[0029] Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

[0030] In a possible implementation, the scheduling code and the subtask file are formed by the following method:

[0031] Get a computational task;

[0032] Splitting the computing task to obtain at least two computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask; the execution processor triggers the execution of the computing subtask based on the completion of the processing of the computing subtask by the corresponding pre-order processor, and the pre-order processor corresponding to each computing subtask is the execution processor corresponding to the previous computing subtask;

[0033] Generate a corresponding subtask file for each computing subtask, and the pre-processor and the execution processor respectively execute the corresponding computing subtask according to the corresponding subtask file;

[0034] A corresponding scheduling code is generated according to the pre-order processor and the execution processor of each computing subtask, and a scheduling code corresponding to each computing subtask is obtained.

[0035] In a possible implementation, splitting the computing task to obtain at least two computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask includes:

[0036] Obtaining computing parameters of each of at least two processors;

[0037] According to the calculation parameters corresponding to each calculation step in the calculation task, the calculation task is divided into at least two calculation subtasks, and each calculation subtask includes at least one calculation step;

[0038] The processor matching the calculation parameters is used as the execution processor of the corresponding calculation subtask;

[0039] Based on the calculation order of each calculation subtask, a preceding processor of each calculation subtask is determined.

[0040] In a possible implementation, generating corresponding scheduling codes according to the pre-order processor and the execution processor of each computing subtask to obtain a preset number of scheduling codes includes:

[0041] determining a coding position corresponding to each of at least two processors;

[0042] Setting a first target code position in a first part of codes in the scheduling code corresponding to each computing subtask to a first value, and setting a non-first target code position in the first part of codes to a second value, wherein the first target code position is a code position corresponding to a pre-order processor of the computing subtask;

[0043] The second target coding position in the second part of the scheduling code corresponding to the computing subtask is set to a first value, and the non-second target coding position in the second part of the code is set to a second value, where the second target coding position is the coding position corresponding to the execution processor of the computing subtask.

[0044] A second aspect of the present application provides a heterogeneous computing processing device, including:

[0045] A receiving module, configured to receive a task completion signal sent by the first processor, wherein the task completion signal indicates that the first processor has completed the first computing subtask;

[0046] An acquisition module, used for acquiring a target scheduling code based on the task completion signal;

[0047] A generating module, used for generating a task starting signal according to the task completion signal and the target scheduling code;

[0048] A trigger module is used to trigger the second processor to execute a second computing subtask according to the subtask file based on the task start signal, and the second computing subtask and the first computing subtask belong to the same computing task.

[0049] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the heterogeneous computing processing method of the first aspect or any implementation of the first aspect.

[0050] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0051] The memory is used to store computer programs;

[0052] The processor is used to execute the computer program so that the electronic device can implement the heterogeneous computing processing method of the above-mentioned first aspect or any implementation manner of the first aspect.

[0053] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the heterogeneous computing processing method of the above-mentioned first aspect or any implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0055] Figure 1 It is a schematic diagram of the existing heterogeneous computing architecture;

[0056] Figure 2 It is a schematic diagram of a process of a heterogeneous computing processing method provided in an embodiment of the present application;

[0057] Figure 3 It is a schematic diagram of a task execution file provided in an embodiment of the present application;

[0058] Figure 4 It is a partial schematic diagram of a SOC composition structure of a heterogeneous computing processing method provided by an embodiment of the present application;

[0059] Figure 5 It is a flow chart of generating a task start signal according to the task completion signal and the target scheduling code provided in an embodiment of the present application;

[0060] Figure 6 It is a partial schematic diagram of another SOC composition structure of the heterogeneous computing processing method provided by the embodiment of the present application;

[0061] Figure 7 It is a schematic diagram of the process of forming a scheduling code and a subtask file provided in an embodiment of the present application;

[0062] Figure 8 This is a schematic diagram of splitting a computing task to obtain a task execution code provided by an embodiment of the present application;

[0063] Fig. 9 It is a flowchart of splitting the computing task to obtain a preset number of computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask, provided in an embodiment of the present application;

[0064] Fig.10 is a schematic diagram of a split computing task provided in an embodiment of the present application;

[0065] Fig.11 This is another schematic diagram of splitting a computing task to obtain a task execution code provided by an embodiment of the present application;

[0066] Fig.12 is another schematic diagram of the splitting computing task provided in an embodiment of the present application;

[0067] Fig.13 It is a flowchart of splitting the computing task to obtain a preset number of computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask, provided in an embodiment of the present application;

[0068] Fig.14 This is a schematic diagram of computing subtasks obtained by splitting a computing task provided in an embodiment of the present application;

[0069] Fig.15 is another schematic diagram of splitting computing tasks provided in an embodiment of the present application;

[0070] Fig.16 It is a flowchart of generating corresponding scheduling codes according to the pre-order processor and the execution processor of each computing subtask to obtain a preset number of scheduling codes provided by an embodiment of the present application;

[0071] Fig.17 It is a schematic diagram of an application scenario of a heterogeneous computing method provided by this application;

[0072] Fig.18 This is a schematic diagram of a heterogeneous computing processing device provided in an embodiment of the present application.

[0073] Fig.19 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0075] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0076] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0077] Figure 2 It is a schematic diagram of a heterogeneous computing processing method provided in an embodiment of the present application. Figure 2 As shown, a heterogeneous computing processing method provided in an embodiment of the present application may include steps 201 to 204, and these steps are described in detail below.

[0078] 201. Receive a task completion signal sent by a first processor, where the task completion signal indicates that the first processor completes a first computing subtask;

[0079] Among them, the heterogeneous computing processing method provided in the embodiment of the present application is applied to a heterogeneous SOC (System on Chip) having multiple processors of different types. The SOC is also provided with a scheduling device for executing the above-mentioned heterogeneous computing processing method. The scheduling device implements the scheduling and execution of heterogeneous computing in multiple processors, and the scheduling device is not a structure in the CPU and does not need to occupy the computing power of the CPU.

[0080] Among them, the SOC has multiple processors, each processor executes the computing subtask assigned to it, and generates a task completion signal when the processor completes the computing subtask.

[0081] Among them, the signal output end of each processor is connected to the scheduling device through a circuit to form a transmission path for the task completion signal, and the signal output end of the processor outputs the task completion signal to the scheduling device through the transmission path for the task completion signal.

[0082] The signal output terminal may be a pin or a combination of multiple pins.

[0083] The task completion signal may be a signal in a format agreed upon between each processor and the scheduling device, such as a high or low level signal.

[0084] As an example, the task completion signal is a high-level pulse signal. After a processor completes its computing subtask, it outputs a high-level pulse signal through its signal output terminal. Correspondingly, after receiving the high-level pulse signal, the scheduling device can determine that the first processor has completed its computing subtask.

[0085] 202. Obtain a target scheduling code based on the task completion signal;

[0086] A computing task is divided into a plurality of computing subtasks in advance, and a scheduling code is preset for each computing subtask, wherein the scheduling code represents information such as an execution processor and a preceding processor of the computing subtask.

[0087] Among them, one scheduling code corresponds to one computing subtask, and each computing subtask constitutes a computing task.

[0088] In a possible implementation, the scheduling codes of each computing subtask in the computing task may be stored in the form of a code queue, and the process of obtaining the target scheduling code corresponding to the scheduling code storage method is described in detail in subsequent embodiments.

[0089] In a possible implementation, the scheduling codes of each computing subtask in the computing task may be stored in the form of a linked list, and the process of obtaining the target scheduling code corresponding to the scheduling code storage method is described in detail in subsequent embodiments.

[0090] After receiving the task completion signal of the first computing subtask, the corresponding target scheduling code is obtained according to the task completion signal of the first computing subtask.

[0091] Among them, the first computing subtask is a predecessor computing subtask of the second computing subtask, and the scheduling code can characterize the execution order of the first computing subtask and the second computing subtask. After the first computing subtask is completed, the corresponding target scheduling code is obtained, and then the execution of the second computing subtask is triggered based on the target scheduling code. The subsequent step 204 describes in detail the process of determining the second computing subtask.

[0092] In a possible implementation, the preceding computing subtask of a computing subtask may be one or more. If the preceding computing subtask of the second computing subtask is one, step 202 is triggered upon receiving a task completion signal of the preceding computing subtask; if the preceding computing subtask of the second computing subtask is more than one, step 202 is triggered upon receiving task completion signals of all preceding computing subtasks.

[0093] 203. Generate a task start signal according to the task completion signal and the target scheduling code;

[0094] Among them, the target scheduling code contains relevant information of the second processor that executes the next computing subtask. Accordingly, after receiving the task completion signal of the first computing subtask which is the preceding computing subtask, the task completion signal and the target scheduling code are combined to generate a task start signal to start the second processor of the second computing task.

[0095] In a possible implementation, the scheduling code includes two parts. The first part of the code is used to characterize the pre-processor of the corresponding computing subtask, and the second part of the code is used to characterize the execution processor of the corresponding subtask. According to the scheduling code, the corresponding execution processor can be determined, and then a task start signal is generated to trigger the execution processor to execute the action. Figure 5 The process of generating a task start signal based on a task completion signal and a target scheduling code is described in detail.

[0096] The task start signal may be a signal in a format agreed upon between each processor and the scheduling device, such as a high or low level signal.

[0097] The task start signal and the task completion signal may be signals of the same type or signals of different types, and may be set according to actual conditions.

[0098] In one possible implementation, in a SOC having multiple processors, a signal input terminal of each processor is connected to the scheduling device through a circuit to form a transmission path for the task start signal, and the scheduling device sends a task start signal to the signal input terminal of the processor through the transmission path for the task start signal.

[0099] 204. Triggering the second processor to execute a second computing subtask according to the subtask file based on the task start signal, where the second computing subtask and the first computing subtask belong to the same computing task.

[0100] The task start signal is for the second processor that executes the second computing subtask. Accordingly, after the task start signal is generated, it is sent to the second processor to trigger the second processor to execute the computing subtask.

[0101] A subtask file is set for each computing subtask, and the subtask file contains various information required for an execution processor that executes the corresponding computing subtask to execute the computing subtask.

[0102] In a possible implementation, the subtask file adopts a task execution file (ELF, Executable and Linkable Format), which includes the input data address, result output address and function (algorithm) of the execution processor. The execution processor performs actions according to the ELF file to implement the execution of the calculation subtask.

[0103] The subtask file may be a file generated for each computing subtask obtained by splitting the computing task in advance, and the subtask file is saved to the execution processor of the corresponding computing subtask.

[0104] In a possible implementation, the required information of each computing subtask executed by a processor forms a task execution file. In the task execution file, the required information of each computing subtask is separated by outputting a task completion signal and waiting for receiving a task start signal.

[0105] Correspondingly, the processor executes the corresponding computing subtask according to the information required to execute the computing subtask in the task execution file. When the computing subtask is completed, the task completion signal is output. The processor pauses and waits for the task start signal to be received, and then executes the computing subtask according to the information required for the subsequent execution of the computing subtask.

[0106] Figure 3 It is a schematic diagram of a task execution file provided by an embodiment of the present application, including relevant information of two computing subtasks, a function statement for outputting a task completion signal, a function statement for waiting for receiving a task start signal, etc. In this schematic diagram, the execution processor of the task execution file is an NPU, and the corresponding information required for executing the computing subtask adopts the sbugraph_npu_elf.bin file. In this schematic diagram, the relevant information of the two computing subtasks adopts sbugraph0_npu_elf.bin and sbugraph1_npu_elf.bin respectively, wherein an output task completion signal (output stop signal) is set after the relevant information of each computing subtask and a wait for receiving a task start signal (wait input start) is set before the relevant information of the next computing subtask.

[0107] Figure 44 is a partial schematic diagram of a SOC structure for applying a heterogeneous computing processing method provided in an embodiment of the present application. The SOC includes four processors 401-404 and a scheduling device 405. Figure 4 In the diagram, the processors include NPU, GPU, DSP and CPU, etc. There are transmission paths for task completion signals and task start signals between each processor and the scheduling device. In the diagram, the transmission path for task completion signals is marked with STOP, and the transmission path for task start signals is marked with START.

[0108] Among them, the scheduling device 405 executes the heterogeneous computing processing method provided in the embodiment of the present application, which is a control for each processor to perform heterogeneous computing. Taking NPU as an example, after the NPU completes its computing subtask, it sends a task completion signal to the scheduling device through the transmission path of the task completion signal, and the NPU pauses and waits, and does not execute subsequent computing subtasks. After receiving the task completion signal, the scheduling device obtains a scheduling code based on the task completion signal. The scheduling code is characterized by the execution processor of the next computing subtask is a GPU, and then sends a task start signal to the GPU through the transmission path of the task start signal between the GPU. Based on the received task start signal, the GPU triggers the start of the next computing subtask. After the GPU completes its computing subtask, it sends a task completion signal to the scheduling device through the transmission path of the task completion signal, and the GPU pauses and waits, and does not execute subsequent computing subtasks. Execute in a loop until all scheduling codes in the scheduling device are used and the computing task is completed.

[0109] In a possible implementation, the computing subtask can be split into multiple computing subtasks in advance, and a corresponding subtask file and scheduling code are generated for each computing subtask. Figure 7 The process of forming scheduling codes and subtask files is described in detail.

[0110] In this embodiment, a task completion signal sent by a first processor is received, and the task completion signal indicates that the first processor completes a first computing subtask; a target scheduling code is obtained based on the task completion signal; a task start signal is generated based on the task completion signal and the target scheduling code; and a second processor is triggered to execute a second computing subtask according to a subtask file based on the task start signal. The second computing subtask and the first computing subtask belong to the same computing task. The computing task is first split into individual computing subtasks, and a scheduling code is set for each computing subtask. The target scheduling code is obtained by generating a task completion signal when the first processor completes a preceding computing subtask. A task start signal is generated based on the target scheduling code and the task completion signal. The second processor executes a subsequent second computing subtask according to the subtask file. A scheduling device is set to execute the scheduling of computing subtasks between the processors without occupying the CPU, thereby reducing the software complexity of the CPU. After reducing the participation of the CPU, the overall power consumption of the SOC is reduced. Moreover, by implementing the inter-core synchronization logic of each processor through encoding, the parallel capability of each processor can be improved, and the computing efficiency of the SOC in executing computing tasks can be improved.

[0111] Figure 5 It is a flowchart of generating a task start signal based on the task completion signal and the target scheduling code provided in an embodiment of the present application, which may include steps 501 to 504, and these steps are described in detail below.

[0112] 501. Parse the target scheduling code to obtain a first part code and a second part code of the target scheduling code;

[0113] The scheduling code includes two parts: a first part code and a second part code.

[0114] The first part of the code represents the preamble processor of the computing subtask, and the second part of the code represents the execution processor of the computing subtask.

[0115] In the first part of the coding and the second part of the coding of the scheduling coding, each processor is encoded according to a set order in the coding.

[0116] Different numerical values ​​may be used in the encoding to indicate whether a processor is used or not.

[0117] In a possible implementation, a code of 1 represents use of the processor, and a code of 0 represents not use of the processor.

[0118] As an example, the SOC includes four processors, namely, NPU, GPU, DSP and CPU. Accordingly, the two parts in the scheduling code correspond to four coding positions, such as xxxxyyyy. Among them, xxxx is the first part of the code, yyyy is the second part of the code, the xxxx represents the pre-processor of the computing subtask, and yyyy represents the execution processor of the computing subtask. The four coding positions of the xxxx correspond to the NPU, GPU, DSP and CPU from left to right, and the four coding positions of yyyy correspond to the NPU, GPU, DSP and CPU from left to right. For example, when the pre-processor of the computing subtask is the NPU and the execution processor is the GPU, the first part of the code xxxx is 1000, the second part of the code yyyy is 0100, and the scheduling code of the computing subtask is 10000100.

[0119] In a possible implementation, the code 11 represents the use of the processor, and the codes 10, 01, and 00 may all represent the non-use of the processor.

[0120] As an example, the SOC includes three processors, NPU, GPU and CPU. Correspondingly, the two parts in the scheduling code correspond to six coding positions, such as aaaaaabbbbbb. Among them, aaaaaa is the first part of the code, bbbbbb is the second part of the code, the aaaaaa represents the pre-processor of the computing subtask, and bbbbbb represents the execution processor of the computing subtask. The six coding positions of aaaaaa correspond to the NPU, GPU and CPU in groups of two from left to right, and the six coding positions of bbbbbb correspond to the NPU, GPU and CPU in groups of two from left to right. For example, when the pre-processor of the computing subtask is the NPU and the execution processor is the GPU, the first part of the code aaaaaa is 110110, the second part of the code bbbbbb is 001011, and the scheduling code of the computing subtask is 110110001011.

[0121] In a possible implementation, when a computing task can be split into multiple computing subtasks, a scheduling code of the computing subtask is generated for the pre-processor and the execution processor of each computing subtask. Fig.16 The process of generating the scheduling code is described in detail.

[0122] 502. Determine, based on the first part of the code and the task completion signal, that each pre-order processor completes each corresponding first computing subtask, the pre-order processor is a processor for executing the first computing subtask, the first computing subtask is a pre-order subtask of the second computing subtask, and the pre-order processor of the second computing subtask includes at least one first processor;

[0123] In this application, multiple first computing subtasks may exist simultaneously before the second subtask, each first computing subtask is executed by a different processor, and the first computing subtask is a predecessor subtask of the second computing subtask, and the second computing subtask is executed based on the calculation results of each first computing subtask.

[0124] Among them, when each preceding processor completes its corresponding first computing subtask, it generates a corresponding task completion signal. Correspondingly, the scheduling device receives the task completion signal through the signal transmission path between each processor. The scheduling device can determine that each preceding processor completes the corresponding first computing subtask based on the target scheduling code and the task completion signal transmitted by each processor.

[0125] As an example, the first part of the scheduling code of the computing subtask is 1100, and the order of the four processors corresponding to the SOC is NPU, GPU, DSP and CPU. Only when the task completion signals transmitted by the NPU and GPU are received respectively, it is determined that all the previous subtasks are completed, triggering step 503, otherwise, wait.

[0126] 503. Determine an execution processor of the second computing subtask according to the second part of the code;

[0127] Wherein, based on the first part of the target scheduling code, it is determined that all preceding processors have completed their corresponding first computing subtasks, and the execution processor of the second computing subtask is determined according to the second part of the target scheduling code.

[0128] In the second part of the coding, for each processor, the execution processor corresponding to the specific value is determined according to the set order.

[0129] As an example, a specific value of the execution processor may be 1, and the values ​​of the execution processor in the remaining codes may be 0.

[0130] As an example, the SOC includes four processors, namely, NPU, GPU, DSP and CPU. The scheduling code of the computing subtask is 11000010. The NPU and GPU are received as predecessor processors. After receiving the task completion signal transmitted by the NPU and GPU, it is determined that all the predecessor subtasks of the computing subtask are completed, and the DSP of the execution processor of the computing subtask is determined.

[0131] As an example, the SOC includes four processors: NPU, GPU, DSP and CPU. The scheduling code of the computing subtask is 00010110. The CPU acts as a preceding processor. Upon receiving the task completion signal transmitted by the CPU, it determines that all preceding subtasks of the computing subtask are completed, and determines the GPU and DSP of the execution processor of the computing subtask.

[0132] 504. Generate a task start signal according to the execution processor corresponding to the second part of the code, where the task start signal is used to trigger the execution processor to execute the second computing subtask.

[0133] Among them, the execution processor of the second computing subtask is determined, a task start signal is generated, and the task start signal is sent to the execution processor through a signal transmission path between the execution processor to trigger the execution processor to execute its second computing subtask.

[0134] Among them, the output logic of the task start signal requires that all preceding subtasks be completed, and task completion signals sent by the first processors corresponding to all preceding subtasks are received to determine that all preceding subtasks are completed; the corresponding execution processor is determined based on the second part of the target scheduling code, and the task start signal is output to the execution processor, thereby realizing synchronous judgment and triggering of the execution processor.

[0135] Figure 6 is a partial schematic diagram of another SOC structure for applying a heterogeneous computing processing method provided in an embodiment of the present application. The SOC includes a scheduling device 601 and four processors 602-605. Figure 6 In the diagram, the processors include NPU, GPU, DSP and CPU. Each processor has a transmission path for task completion signal and a transmission path for task start signal between it and the scheduling device. In the diagram, the transmission path for task completion signal is marked with STOP, and the transmission path for task start signal is marked with START. The direction of the arrow in the diagram indicates the direction of signal transmission.

[0136] The scheduling device 601 is provided with scheduling codes corresponding to each computing subtask. The scheduling device includes a logic unit 6011 and a coding unit 6012 . The logic unit includes a coding logic circuit 60111 and an output logic circuit 60112 .

[0137] The encoding unit stores the scheduling codes of each computing subtask. The encoding logic circuit is connected to each processor to form a transmission path for the task completion signal. After receiving the task completion signal, the encoding logic circuit 60111 generates a POP signal corresponding to the processor executing the task, and the POP signal indicates the processor corresponding to the task completion signal in a specific signal encoding method. The encoding unit 6012 outputs the scheduling code to the output logic circuit 60112 based on the POP signal. After receiving the scheduling code, the output logic circuit 60112 generates a task start signal to the corresponding execution processor based on the scheduling code.

[0138] Among them, the encoding unit 6012 can be DDR / SRAM, can be a small piece of storage configured separately for the scheduling device, or can reuse other storage resources of the soc chip, which is not limited in this application.

[0139] In this embodiment, the target scheduling code is parsed to obtain the first part code and the second part code of the target scheduling code; based on the first part code and the task completion signal, it is determined that each pre-order processor completes the corresponding first computing subtasks, and the pre-order processor is a processor for executing the first computing subtask, the first computing subtask is a pre-order subtask of the second computing subtask, and the pre-order processor of the second computing subtask includes at least one first processor; based on the second part code, the execution processor of the second computing subtask is determined; based on the execution processor corresponding to the second part code, a task start signal is generated, and the task start signal is used to trigger the execution processor to execute the second computing subtask. In the scheduling code, a preamble processor code for executing the preamble first computing subtask and an execution processor code for executing the second computing subtask are respectively set. Accordingly, based on the preamble processor code and the received task completion signal, it is determined that each preamble processor completes the corresponding first computing task, which can trigger the execution processor of the second computing subtask, and then a task start signal corresponding to each execution processor is generated to trigger and start each execution processor to execute the second computing task. In this process, based on the task completion signal of the first processor of the preamble subtask, it is determined whether all the first part of the code in the target scheduling code is satisfied, and the corresponding execution processor is determined based on the second part of the code in the target scheduling code, and the task start signal is output to the execution processor, thereby realizing synchronous judgment and triggering of the execution processor.

[0140] In a heterogeneous computing processing method provided in an embodiment of the present application, it is also necessary to store the calculation result of the first computing subtask so that the second processor can subsequently execute the second computing subtask based on the calculation result of the first computing subtask.

[0141] In a possible implementation, after step 104, the method further includes:

[0142] Control the second processor to obtain the execution result of the first computing subtask according to the storage address corresponding to the subtask file in the preset storage area, so as to execute the second computing subtask according to the execution result of the first computing subtask, and the preset storage area is pre-allocated with a storage address for each computing subtask.

[0143] When a computing task is split into multiple computing subtasks, storage locations for execution results of the various computing subtasks are allocated.

[0144] In the subtask file corresponding to each computing subtask, the storage address of the previous subtask and the storage address of the current computing subtask are set.

[0145] In a possible implementation, the preset storage area may be a storage device such as SRAM (Static Random Access Memory) / DDRSRAM (Double Data Rate SRAM), and the storage address of the calculation results of each calculation subtask is pre-set in the preset storage area.

[0146] Among them, the first processor executes the first computing subtask and obtains an execution result, and stores the execution result in a storage address allocated to the first computing subtask in a preset storage area; when the second processor subsequently executes the second computing subtask, based on the storage address recorded in the subtask file, obtains the execution result of the first computing subtask from the corresponding storage address in the preset storage area, and the second processor executes the second computing subtask based on the execution result of the first computing subtask, and stores the execution result in the storage address allocated to the second computing subtask in the preset storage area.

[0147] In this embodiment, a storage address is pre-allocated to each computing subtask in the preset storage area. When the second processor is triggered to start executing the second computing subtask, the second processor is controlled to obtain the execution result of the first computing subtask according to the storage address corresponding to the subtask file in the preset storage area. The second processor executes the second computing subtask according to the execution result of the first computing subtask, thereby realizing the application of the execution result of the previous subtask to the computing subtask.

[0148] In one possible implementation, obtaining a target scheduling code based on a task completion signal includes: determining that the first first computing subtask corresponding to the second computing subtask is completed based on the task completion signal, and obtaining a target scheduling code in a local preset scheduling code queue in response to the task completion signal.

[0149] Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes:

[0150] In response to the task completion signal, determining the completion status of each first computing subtask;

[0151] Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

[0152] Among them, after receiving a task completion signal from a processor that completes the execution of a computing subtask, each computing subtask corresponding to the previous scheduling code has been completed, and the task completion signal is the completion of the first computing subtask as a predecessor subtask of the second computing subtask, and the first computing subtask corresponding to the task completion signal is the first completed predecessor subtask of the second computing subtask.

[0153] In one possible implementation, each time a task completion signal is received, a target scheduling code can be obtained in the scheduling code queue according to the task completion signal, until all the preceding processors in the target scheduling code complete their computing subtasks.

[0154] However, in the above process, the process of obtaining the target scheduling code needs to be repeated, which places a heavy burden on the scheduling device.

[0155] Therefore, in this embodiment, after receiving a task completion signal, a scheduling code acquisition process can be triggered. Subsequently, when a task completion signal is received, if the task completion signal is generated by a preceding processor in the scheduling code, the scheduling code acquisition process will not be triggered again until each preceding processor in the scheduling code completes the subtask, and a task start signal is generated based on the execution processor recorded in the target scheduling code to start the execution processor to execute the second computing subtask.

[0156] Wherein, a scheduling code queue is pre-set locally in the scheduling device, and the scheduling code of each computing subtask is put into the scheduling code queue according to the order in the computing task.

[0157] Among them, the logic unit in the scheduling device can determine that the first computing subtasks of each preceding order are completed based on the task completion signal, and control to pop a scheduling code from the scheduling code queue, and the scheduling code is the target scheduling code.

[0158] Among them, the scheduling code queue can be that when splitting the computing subtasks, after generating the scheduling code for each computing subtask, the scheduling code of the computing subtask is pushed into the scheduling code queue according to the sequence of each computing subtask in the computing task.

[0159] In a possible implementation, the scheduling encoding queue may adopt a FIFO (First Input First Output) queue.

[0160] Among them, the scheduling codes in the scheduling code queue are stored according to the execution order of each computing subtask, specifically in a first-in-first-out order, and the computing subtasks with an earlier execution order are pushed into the scheduling code queue first, and the computing tasks with a later execution order are pushed into the scheduling code queue later.

[0161] As an example, the SOC includes four processors, namely, NPU, GPU, DSP and CPU, among which three computing subtasks need to be executed sequentially. The execution processor of computing subtask 1 is NPU, the execution processor of computing subtask 2 is DSP and CPU, and the execution processor of computing subtask 1 is GPU. Accordingly, since computing subtask 1 has no preceding subtask, the scheduling code of computing subtask 1 is 00001000, the scheduling code of computing subtask 2 is 10000011, and the scheduling code of computing subtask 3 is 00110100. The scheduling codes 00001000, 10000011 and 00110100 are sequentially pushed into the scheduling code queue. Accordingly, when the scheduling codes in the scheduling code queue are popped out, they are also popped out in the order of 00001000, 10000011 and 00110100.

[0162] When the depth of the scheduling code queue is not less than the number of scheduling codes of computing subtasks in the computing task, all scheduling codes can be stacked into the scheduling code queue without temporarily storing the scheduling codes from the external storage structure.

[0163] Among them, when the depth of the scheduling code queue is less than the number of scheduling codes of the computing subtasks in the computing task, the scheduling codes that exceed the scheduling code queue depth need to be stored in an external preset memory first, and during the execution of each computing subtask, the scheduling codes are read from the preset memory according to predetermined rules.

[0164] The predetermined rule may include popping a scheduling code from a scheduling code queue, and obtaining a scheduling code from the external preset memory and pushing it into the scheduling code queue.

[0165] The preset memory may be a storage structure such as DDR.

[0166] Among them, the location address of the scheduling code stored in the external storage structure is pre-specified, and correspondingly, the address location of the scheduling code in the external storage structure and information such as the specific number obtained are pre-configured, so that during the execution of each computing subtask, a specific number of scheduling codes can be obtained from the corresponding address location in the external storage structure.

[0167] As an example, the external storage structure may adopt a storage structure such as SRAM or DDR, such as the memory of a SOC chip.

[0168] refer to Figure 6 In the SOC schematic diagram of the heterogeneous computing processing method shown in , the encoding unit 6012 in the scheduling device can use the form of a queue to save the scheduling codes of each computing subtask. After a scheduling code in the encoding unit is popped, the logic unit in the scheduling device receives a task completion signal, which is sent by the processor of the first computing subtask, triggering the next scheduling code in the encoding unit to be popped, and obtaining the target scheduling code. The logic unit subsequently continues to receive task completion signals sent by each processor, compares the processor from which each task completion signal comes with the first part of the target scheduling code, and waits for the task completion signals of all processors corresponding to the first part of the scheduling code to be received, generates a task start signal and sends it to the processor in the second part of the scheduling code, so as to trigger the execution of the next computing subtask.

[0169] In a possible implementation, the application heterogeneous computing processing method further includes the following steps after popping the scheduling code from the scheduling code queue:

[0170] Based on the idle state in the scheduling code queue, at least one scheduling code is read from the scheduling codes stored in the preset memory and put into the scheduling code queue, and the computing subtasks corresponding to the scheduling codes stored in the preset memory constitute a computing task.

[0171] Among them, the number of scheduling codes is the same as the number of computing subtasks, and the number of computing subtasks can be determined based on the number of computing subtasks actually obtained by splitting the computing tasks. This application does not limit the number of computing subtasks.

[0172] As an example, the preset memory may adopt a storage structure such as DDR.

[0173] If the depth of the scheduling code queue is less than the total number of scheduling codes of computing subtasks in the computing task, some scheduling codes that are sorted later are stored in a preset memory in advance. During the execution of the computing task, when the scheduling codes in the scheduling code queue are popped out, the scheduling code queue becomes idle, and the scheduling code is read from the scheduling codes stored in the preset memory. The scheduling code read from the preset memory is sorted after the last scheduling code in the scheduling code queue.

[0174] Furthermore, the read scheduling code is stacked in the scheduling code queue to implement stacking and popping of the FIFO queue.

[0175] Among them, if the depth of the scheduling code queue is not less than the total number of scheduling codes of computing subtasks in the computing task, during the execution of the computing task, when the scheduling code in the scheduling code queue is popped out, even if the scheduling code queue becomes idle, there is no need to add scheduling codes to the scheduling code queue.

[0176] In this embodiment, a scheduling code queue is preset locally in the scheduling device, and when the completion of the first first computing subtask is determined based on the task completion signal, in response to the first computing subtask, and when all the first computing subtasks are completed, a task start signal is generated according to the target scheduling code to trigger the execution of the second computing subtask, and also based on the idle state in the scheduling code queue, one or more scheduling codes are read from the preset number of scheduling codes stored in the preset memory and put into the scheduling code queue, and the computing subtasks corresponding to the preset number of scheduling codes stored in the preset storage constitute a computing task, and when the computing task is split into multiple computing subtasks, the generated scheduling code is pre-stored in the preset memory, and as the computing subtask is executed, the scheduling code is moved from the preset memory to the preset scheduling code queue according to the idle state of the scheduling code queue, thereby ensuring that even when the depth of the scheduling code queue is small, all scheduling codes corresponding to the computing task can be put into the scheduling code queue.

[0177] In a possible implementation, obtaining a target scheduling code based on a task completion signal includes: determining that a first computing subtask is completed based on the task completion signal, and determining a target scheduling code corresponding to a second computing subtask subsequent to the first computing subtask according to a preset linked list sequence;

[0178] Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes:

[0179] In response to the task completion signal, determining the completion status of each first computing subtask;

[0180] Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

[0181] Wherein, in the scheduling device, the scheduling code is stored in a linked list structure, and the preset linked list is used to record the storage location of the scheduling code corresponding to each computing subtask.

[0182] The linked list is a common data structure, which is composed of a series of nodes, each of which contains two parts: a data part and a pointer to the next node. Accordingly, in this embodiment, the data part in the node is a scheduling code.

[0183] The preset linked list may be a list in which the scheduling codes of each computing subtask are generated when the computing subtasks are split, and the linked list is recorded in the logic unit of the scheduling device after the scheduling codes of each computing subtask are stored in the form of a linked list.

[0184] Among them, a storage area can be set in the scheduling device, and each node of the linked list is stored in the set storage area. The data field in the node is used to store the scheduling code of the preceding calculation subtask, and the pointer field in the node is used to store the node address of the subsequent calculation subtask.

[0185] Among them, after receiving a task completion signal from a processor that completes the execution of a computing subtask, each computing subtask corresponding to the previous scheduling code has been completed, and the task completion signal is the completion of the first computing subtask as a predecessor subtask of the second computing subtask, and the first computing subtask corresponding to the task completion signal is the first completed predecessor subtask of the second computing subtask.

[0186] In one possible implementation, each time a task completion signal is received, the target scheduling code can be obtained according to the pointer of the preset list triggered by the task completion signal until all the preceding processors in the target scheduling code complete their computing subtasks.

[0187] However, in the above process, the process of obtaining the target scheduling code needs to be repeated, which places a heavy burden on the scheduling device.

[0188] Therefore, in this embodiment, after receiving a task completion signal, a scheduling code acquisition process can be triggered. Subsequently, when a task completion signal is received, if the task completion signal is generated by a preceding processor in the scheduling code, the scheduling code acquisition process will not be triggered again until each preceding processor in the scheduling code completes the subtask and generates a task start signal to be sent to the processor in the second part of the scheduling code, so as to start the execution processor to execute the second computing subtask.

[0189] refer to Figure 6In the SOC schematic diagram of the heterogeneous computing processing method shown in , the coding unit in the scheduling device can use a linked list to save the scheduling codes of each computing subtask. The logic unit in the scheduling device obtains a scheduling code in the coding unit. After receiving the task completion signal, the task completion signal is sent by the pre-order processor corresponding to the first computing subtask, and the pointer of the node where the scheduling code of the current first computing subtask is located is obtained to determine the node address of the subsequent computing subtask, and the target scheduling code is obtained from the data part corresponding to the node address. The processors corresponding to each subsequent task completion signal received are compared according to the first part of the target scheduling code, and the task completion signals of all processors corresponding to the first part of the target scheduling code are received. A task start signal is generated and sent to the execution processor in the second part of the target scheduling code to trigger the execution of the subsequent computing subtask.

[0190] In this embodiment, a linked list is preset in the scheduling device, and the linked list records the scheduling code of the preceding computing subtask and the node address of the succeeding computing subtask. Accordingly, based on the task completion signal, it is determined that each first computing subtask is completed, and the target scheduling code corresponding to the second computing subtask that is succeeding the first computing subtask is determined according to the preset linked list. Based on the preset linked list, the sequential execution of each computing subtask in the computing task is realized.

[0191] Figure 7 It is a flowchart of forming a scheduling code and a subtask file provided in an embodiment of the present application, which may include steps 701 to 704. These steps are described in detail below.

[0192] 701. Obtain computing tasks;

[0193] The computing task is a complete computing task for subsequent splitting, and the computing task includes multiple computing steps. The computing power required for each computing step may be different, and the data formats of different computing steps may also be different.

[0194] If the computing task can be a built-in function in the SOC, the computing task can be split in advance to obtain a scheduling code and a subtask file, and the obtained scheduling code and subtask file can be put into the SOC.

[0195] If the computing task is in a third-party APP (application), the computing task therein may be encoded locally in the SOC when the application is loaded for the first time.

[0196] In a possible implementation, the scheduling device may split the computing task in an offline state after obtaining the computing task. For example, after the scheduling device loads a third-party APP online, it splits the computing task in the APP offline.

[0197] In a possible implementation, after obtaining the computing task, the scheduling device may split the computing task in an online runtime state. For example, when the scheduling device loads a third-party APP online, the scheduling device directly splits the computing task in the APP.

[0198] As an example, the computing subtask may be various types of computing tasks, such as computing tasks of AI (Artificial Intelligence) models, graphics processing tasks, image rendering tasks, and the like.

[0199] 702. Split the computing task to obtain at least two computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask; the execution processor triggers the execution of the computing subtask based on the completion of the processing of the computing subtask by the corresponding pre-order processor, and the pre-order processor corresponding to each computing subtask is the execution processor corresponding to the previous computing subtask;

[0200] Among them, the computing task can be split into multiple computing subtasks, and the number of the computing subtasks can be the number of computing subtasks actually obtained by splitting the computing task according to the splitting rules. This application does not limit the number of computing subtasks.

[0201] Among them, one computing subtask corresponds to a pre-order processor and an execution processor, the pre-order processor is used to execute the pre-order subtask of the computing subtask, and the execution result of the pre-order processor is used as the input of the execution processor.

[0202] The computing processor is split to obtain a plurality of computing subtasks, and an execution processor for each computing subtask is obtained, and the preceding processor for each computing subtask can be determined according to the order of splitting.

[0203] In a possible implementation, the rule for splitting the computing task can be based on the computing parameters of each processor, splitting the computing task to obtain multiple computing subtasks; it can also be based on the execution order of each step in the computing task, splitting the computing task to obtain multiple computing subtasks; it can also be based on the computing parameters of the processor and the execution order of each step in the computing task, splitting the computing task to obtain multiple computing subtasks, and then Fig. 9 The process of splitting computing tasks based on processor computing parameters is described in detail. Fig.13The process of splitting computing tasks based on the execution order of each step in the computing task is described in detail.

[0204] 703. Generate a corresponding subtask file for each computing subtask, and the pre-order processor and the execution processor respectively execute the corresponding computing subtask according to the corresponding subtask file;

[0205] For each computing subtask obtained by splitting, a corresponding subtask file is generated, and the subtask file may include the input data address, result output address and function (algorithm) of the execution processor.

[0206] The data input address and result output address in each computing subtask may be a storage area preset in the SOC, and the storage area stores the execution result of each computing subtask, and the storage area needs to be resident in the SOC.

[0207] Among them, by specifying the entry for interaction with the storage area, an internal address encryption method, such as PA / VA (Physical Address & Virtual Address), can be adopted in the storage area. The input data address and result output address recorded in the subtask file are physical addresses. The execution processor requests the execution result of the previous subtask from the storage area according to the physical address in the subtask file. The execution result of the previous subtask is obtained in the storage area according to the internal virtual address and fed back.

[0208] Each processor executes its computing subtask according to the subtask file corresponding to the processor.

[0209] Among them, the pre-processor executes its corresponding first computing subtask according to its corresponding subtask file, and the execution processor executes its corresponding second computing subtask according to its corresponding subtask file. The execution result of the first computing subtask is used as the input of the second computing subtask. After the first computing subtask is executed, the execution result is stored in the allocated result output address. The result output address of the first computing subtask is the input data address of the second computing subtask. Correspondingly, the execution processor obtains the execution result of the first computing subtask according to the input data address of the second computing subtask, and executes the second computing subtask according to the execution result of the first computing subtask.

[0210] In a possible implementation, the computing task is the calculation of a neural network model. Accordingly, the neural network compiler TVM can be used to process the split computing subtasks to generate corresponding subtask files.

[0211] In a possible implementation, if the executors of several computing subtasks are the same processor, the subtask files of the several computing subtasks executors of the same processor can be generated into the task execution file ELF of the processor. In the task execution file, the relevant information of each computing subtask is separated by specific statements.

[0212] The specific statement in the task execution file is used to output a task completion signal to pause when a computing subtask is executed, and to start the execution of the next computing subtask when a task start signal is received.

[0213] As an example, the specific statement is output stop signal&wait input start. Among them, stop signal is a task completion signal, output stop signal means outputting a task completion signal after completing the calculation subtask; among them, start is a task start signal, and wait input start means waiting for the task start signal to execute subsequent steps.

[0214] Figure 8 This is a schematic diagram of a method of splitting a computing task to obtain a task execution code provided by an embodiment of the present application. In this schematic diagram, the computing task is split into four computing subtasks 801-804, and the relevant information of each computing subtask is compiled by the neural network compiler TVM to obtain a corresponding subgraph. Among them, computing subtask 801 corresponds to sbugraph0, and the execution processor is NPU; computing subtask 802 corresponds to subgraph1, and the execution processor is GPU; computing subtask 801 corresponds to sbugraph2, and the execution processor is NPU; computing subtask 804 corresponds to subgraph3, and the execution processor is GPU. Correspondingly, the subtask file corresponding to computing subtask 801 is sbugraph0_npu_elf.bin, the subtask file corresponding to computing subtask 802 is sbugraph1_gpu_elf.bin, the subtask file corresponding to computing subtask 803 is sbugraph2_npu_elf.bin, and the subtask file corresponding to computing subtask 804 is sbugraph3_gpu_elf.bin. The computing subtasks of the same processor are merged to obtain the ELF file of the processor. Figure 8The ELF file 805 is the task execution file of the NPU, in which output stop signal & wait input start are added between sbugraph0_npu_elf.bin and sbugraph2_npu_elf.bin. After the NPU completes the calculation subtask 801, it pauses and waits for the task start signal to execute the calculation subtask 803. Similarly, Figure 8 The ELF file 806 is the task execution file of the GPU, wherein output stop signal & waitinput start is added between sbugraph1_gpu_elf.bin and sbugraph3_gpu_elf.bin, and the GPU pauses after completing the calculation subtask 802 and waits for the task start signal to be received before executing the calculation subtask 804. Wherein, after the NPU completes the calculation subtask 801 according to sbugraph0_npu_elf.bin, it generates a task completion signal and pauses; the scheduling device triggers the generation of a task start signal to the GPU according to the task completion signal, and the GPU receives the task start signal, completes the calculation subtask 802 according to sbugraph1_gpu_elf.bin, generates a task completion signal, and pauses; the scheduling device triggers the generation of a task start signal to the NPU according to the task completion signal, and the NPU receives the task start signal and starts to execute the subsequent calculation subtask 803 according to sbugraph0_npu_elf.bin, and so on, and each calculation subtask in the calculation task is executed to obtain the execution result of the calculation task.

[0215] 704. Generate a corresponding scheduling code according to the pre-order processor and the execution processor of each computing subtask to obtain a scheduling code corresponding to each computing subtask.

[0216] After the pre-order processor and the execution processor of each computing subtask are determined, the corresponding scheduling code is generated according to the pre-order processor and the execution processor to obtain the scheduling code corresponding to the computing subtask one by one.

[0217] In a possible implementation, the process of splitting the computing task, generating the subtask files and compiling the scheduling code can be performed when the SOC is offline to reduce the processing burden of the SOC.

[0218] The scheduling code includes a first part of code and a second part of code, the first part of code is used to characterize a preceding processor, and the second part of code is used to characterize an execution processor.

[0219] If there are multiple pre-order processors for a certain computing subtask, each pre-order processor in the first part of the code needs to be represented by a specific numerical value.

[0220] In a possible implementation, a code of 1 represents use of the processor, and a code of 0 represents not use of the processor.

[0221] As an example, the SOC includes four processors: NPU, GPU, DSP and CPU. Correspondingly, the two parts in the scheduling code correspond to four coding positions. If the preceding processor of the computing subtask is NPU and the execution processor is CPU, the corresponding scheduling code is 10000001; if the preceding processor of the computing subtask is NPU and GPU, and the execution processor is CPU, the corresponding scheduling code is 11000001.

[0222] In a possible implementation, a value of a coding position corresponding to the pre-order processor in the first part of the coding is set for the pre-order processor, and a value of a coding position corresponding to the execution processor in the second part of the coding is set for the execution processor, and then Fig.15 The process of generating corresponding scheduling codes based on the pre-order processor and execution processor of each computing subtask is described in detail.

[0223] In this embodiment, a computing task is split to obtain multiple computing subtasks and a preamble processor and an execution processor corresponding to each computing subtask. The preamble processor corresponding to each computing subtask serves as the execution processor corresponding to the previous computing task. A corresponding subtask file is generated for each computing subtask. Each computing subtask executes the corresponding computing subtask according to the corresponding subtask file. In addition, a corresponding scheduling code is generated according to the preamble processor and the execution processor of each computing subtask to obtain scheduling codes corresponding to each computing subtask. The splitting of computing tasks is realized to form scheduling codes and subtask files, thereby providing a basis for the scheduling device to execute heterogeneous computing processing methods.

[0224] Fig. 9 The embodiment of the present application provides a flowchart for splitting the computing task to obtain a preset number of computing subtasks and a preceding processor and an execution processor corresponding to each computing subtask, which may include steps 901 to 904. These steps are described in detail below.

[0225] 901. Obtain computing parameters of each processor of at least two processors;

[0226] In this embodiment, the splitting rule is to split according to the calculation parameters of the processor and the calculation parameters required for the calculation steps in the calculation task.

[0227] The calculation parameters may include the computing capability of the processor or the data types supported for calculation.

[0228] Each processor may be provided with multiple operators, and the computing capability of each operator may be different.

[0229] The data type supported by each processor for calculation may be one or more, and the data type may be INT4, INT8, INT16, or FP32, etc.

[0230] The calculation parameters of the processor may be used as parameters of the processor and written into a relevant parameter record of the processor, and the calculation parameters may be directly obtained from the relevant parameter record.

[0231] Of course, the present application does not limit the source of the calculation parameters of the processor. During the specific implementation, the calculation parameters of each processor can be set according to the actual situation.

[0232] 902. Split the computing task into at least two computing subtasks according to computing parameters corresponding to each computing step in the computing task, wherein each computing subtask includes at least one computing step;

[0233] The computing task includes a plurality of computing steps, and a corresponding computing parameter may be determined for each computing step. The computing parameter may be the computing power required to execute the computing step or the data type of the computing step.

[0234] The calculation can be split according to each calculation step, and each calculation subtask obtained includes a calculation step.

[0235] If the computing power required for some consecutive computing steps is relatively small, these consecutive computing steps can be used as a computing subtask, and the computing subtask matches the computing power of an operator in a processor.

[0236] If the data types of some consecutive calculation steps are the same, these consecutive calculation steps can be used as a calculation subtask, and the data types of these consecutive calculation steps match the data type executed by a processor.

[0237] In one possible implementation, the computing task can be split only according to the computing power of the processor to obtain multiple computing sub-tasks; the computing task can also be split only according to the data type supported by the processor to obtain multiple computing sub-tasks; the computing task can also be split in combination with the computing power of the processor and the supported data type to obtain multiple computing sub-tasks.

[0238] 903. Using the processor that matches the computing parameter as the execution processor of the corresponding computing subtask;

[0239] The computing subtasks obtained by splitting are assigned to processors with matching computing parameters.

[0240] In a possible implementation, the calculation parameters of each processor may be compared and matched with the calculation parameters corresponding to each calculation step, so that each step in the calculation task may be allocated to the processor according to the calculation parameters of the processor.

[0241] Among them, the calculation steps that each processor can support can be determined only by comparing the computing power of the processor with the computing power required for each calculation step; the calculation steps that each processor can support can be determined only by comparing the data type supported by the processor with the data type of each calculation step; and the calculation steps that each processor can support can also be determined by comparing the computing power and supported data types of the processor with the computing power and data types required for each calculation step.

[0242] Among them, when splitting the computing tasks in combination with the computing power of the processor and the supported data type, the computing power of each operator in the processor can be combined with the capacity of the supported data type, and the processor that can support it can be selected as the execution processor from multiple processors; or the data type supported by the processor can be combined with the computing power of each operator therein, and then the processor that can support it can be selected as the execution processor from multiple processors; if multiple processors all support the data type and the operator supports the computing power required for the calculation step, the computing tasks can be split according to the vivid information, and the corresponding processor can be determined as the execution processor.

[0243] 904. Determine a preceding processor for each computing subtask based on the computing order of each computing subtask.

[0244] Except for the first computing subtask, each computing subtask has a preceding processor.

[0245] According to the calculation order of each computing subtask, a pre-order processor of each computing subtask is determined among the multiple computing subtasks obtained by splitting.

[0246] Of course, there may be a situation where a certain computing subtask has multiple preceding subtasks, and then it has multiple preceding processors, and the multiple preceding processors can perform computing in parallel.

[0247] Fig.10It is a schematic diagram of a split computing task provided in an embodiment of the present application. The computing task includes multiple computing steps, and the operators required for each computing step are operator 1, operator 2, operator 1, operator 5, operator 6, etc. Among them, the NPU includes operator 1 and operator 2, and its performance is better than other processors including operator 1 and operator 2. The three computing steps that need to be executed by operator 1, operator 2, and operator 1 are divided into one computing subtask, and the computing subtask is executed by the NPU. Among them, the GPU includes operator 5 and operator 6, and its performance is better than other processors including operator 5 and operator 6. The two computing steps that need to be executed by operator 5 and operator 6 are divided into one computing subtask, and the computing subtask is executed by the GPU.

[0248] Fig.11 This is another schematic diagram of splitting a computing task to obtain a task execution code provided by an embodiment of the present application. In this schematic diagram, the computing task includes multiple computing steps, each box represents a computing step, and the computing task is split into multiple computing subtasks 1101-1103 (three of which are shown in the figure), and the dotted box represents a computing subtask. The split computing subtasks are compiled through the neural network compiler TVM to obtain a subgraph (subgraph) of each computing subtask. The figure takes the four compiled subgraphs (subgraph0-subgraph3) as an example for explanation. Among them, the execution processors of subgraph0 and subgraph2 are NPUs, and the execution processors of subgraph1 and subgraph3 are GPUs. Generate an ELF file 1104 of the NPU, in which output stop signal & wait input start is added between sbugraph0_npu_elf.bin and sbugraph2_npu_elf.bin; generate an ELF file 1105 of the GPU, in which output stop signal & wait input start is added between sbugraph1_gpu_elf.bin and sbugraph3_gpu_elf.bin.

[0249] Fig.12It is another schematic diagram of the split computing task provided in an embodiment of the present application, which includes multiple computing steps, wherein the data types of each computing step are INT8, INT4, INT8, FP16, FP32, etc. Among them, the NPU supports the calculation of data types of INT8 and INT4, and its performance is better than other processors that support INT8 and INT4. The three computing steps using INT8 and INT4 are divided into one computing subtask, and the computing subtask is executed by the NPU. Among them, the GPU supports the calculation of data types of FP16 and FP32, and its performance is better than other processors that support FP16 and FP32. The two computing steps using FP16 and FP32 data types are divided into one computing subtask, and the computing subtask is executed by the GPU.

[0250] In this embodiment, the computing parameters of each processor are not exactly the same. The computing task can be split into multiple computing subtasks based on the computing parameters corresponding to each computing step in the computing task. Each computing subtask includes at least one computing step. According to the computing parameters of each processor, the processor matching each computing subtask is determined, and the processor matching the computing subtask is used as the execution processor of the corresponding computing subtask. The calculation order of each computing subtask is determined, and the pre-order processor of each computing subtask is determined. The splitting of the computing task is realized, and the execution processor and pre-order processor of each computing subtask are determined, which provides a basis for the subsequent generation of scheduling code.

[0251] Fig.13 The embodiment of the present application provides a flowchart for splitting the computing task to obtain a preset number of computing subtasks and a preceding processor and an execution processor corresponding to each computing subtask, which may include steps 1301 to 1303. These steps are described in detail below.

[0252] 1301. Determine the execution order of each step in the computing task;

[0253] In this embodiment, the splitting rule is to split according to the order of each step of the computing task.

[0254] There may be parallel computing steps in the computing task. If there are parallel computing steps, the parallel computing steps can be divided into different processors for execution.

[0255] The execution order of each step in the computing task may be determined first, and the execution order includes parallel and serial.

[0256] Among them, in the parallel calculation step, the calculation results of two or more parallel calculation steps are respectively used as inputs of subsequent adjacent calculation steps; in the serial calculation step, the calculation result of a previous calculation step is used as input of a subsequent adjacent calculation step.

[0257] If there are parallel computing steps in the computing task, execute step 1302 ; if there are no parallel computing steps in the computing task, execute step 1303 .

[0258] 1302. If there are parallel computing steps in the computing task, split the parallel computing steps in the computing task into at least two parallel computing subtasks, and split the serial computing steps in the computing task into at least two serial computing subtasks;

[0259] If there are parallel computing steps in the computing task, the parallel computing part is split to obtain parallel computing subtasks; and the remaining serial computing tasks in the computing task are split into serial computing subtasks.

[0260] Among them, the predecessor subtask of a computing subtask among the multiple computing subtasks may be two or even more parallel computing subtasks, or may be a single computing subtask.

[0261] Fig.14 is a schematic diagram of computing subtasks obtained by splitting the computing task provided in the embodiment of the present application, Fig.14 The computing task 1401 is split into computing subtasks 1-5, wherein computing subtask 2 and computing subtask 3 are parallel computing subtasks, the execution result of computing subtask 1 is used as the input of computing subtask 2 and computing subtask 3, the execution results of computing subtask 2 and computing subtask 3 are used as the input of computing subtask 4, and the execution result of computing subtask 4 is used as the input of computing subtask 5. Among them, computing subtask 4 and computing subtask 5 are serial computing subtasks.

[0262] After executing step 1302 to split into multiple computing subtasks, step 1304 is executed.

[0263] Fig.15 This is another schematic diagram of the split computing task provided by the embodiment of the present application. The computing task includes 15 computing steps, and the operators required for each computing step are operators 1-15 in sequence. Among them, the computing steps corresponding to operators 1-3 are executed by NPU as computing subtask 1, the computing steps corresponding to operators 4-5 are executed by GPU as computing subtask 2, the computing steps corresponding to operators 6-8 are executed by DSP as computing subtask 3, the computing steps corresponding to operators 9-12 are executed by CPU as computing subtask 4, the computing steps corresponding to operator 13 are executed by GPU as computing subtask 5, and the computing steps corresponding to operators 14-15 are executed by NPU as computing subtask 6.

[0264] Among them, the computing subtasks 1-3 are parallel computing subtasks, and the computing subtasks 1-3 and computing subtask 5 are in a serial relationship. The computing subtasks 1-3, 5 and computing subtask 4 are parallel computing subtasks. Accordingly, during the execution process, computing subtasks 1-3 can be executed first, and the computing subtask 5 can be triggered to execute based on the completion of the computing subtasks 1-3. During the execution of computing tasks 1-3 or computing subtask 5, computing subtask 4 can be executed, and after computing subtasks 5 and computing subtask 4 are completed, computing subtask 6 can be triggered to execute.

[0265] 1303. If there is no parallel computing step in the computing task, split the computing task into at least two serial computing subtasks;

[0266] If all the calculation steps in the calculation task are serial and there is no parallel calculation step, then the serial steps of the calculation task can be split into multiple serial calculation subtasks.

[0267] Among them, the preceding computing task of each computing subtask is unique.

[0268] After executing step 1303 to split into multiple computing subtasks, step 1304 is executed.

[0269] 1304. Determine an execution processor for each computing subtask;

[0270] The execution processor of each computing subtask may be determined according to one or more of the computing capability, data type, etc. required by each computing subtask.

[0271] There may be one or more execution processors for each computing subtask, and a corresponding number of processors may be determined from the multiple processors as execution processors for the corresponding computing subtasks according to a determination condition.

[0272] Among them, according to the computing power of the operator in each processor, the supported data type, etc., a matching processor can be determined for multiple computing subtasks as the execution processor of the computing subtask.

[0273] 1305. Determine a preceding processor of each computing subtask based on computing subtasks that are sequentially adjacent to each computing subtask.

[0274] After the execution processor of each computing subtask is determined, the preceding subtask of each computing subtask can be determined according to the order of each computing subtask, and then the preceding processor of each computing subtask can be determined.

[0275] In this embodiment, the execution order of each step in the computing task is first determined. If there is no parallel computing step in the computing task, but only serial computing steps, the computing task is split into multiple serial computing subtasks. If there is a parallel computing step in the computing task, the parallel computing step is split into multiple parallel computing subtasks, and the serial computing step is split into serial computing subtasks. The execution processor of each computing subtask is determined in turn, and then the pre-order processor of each computing subtask is determined based on the computing subtasks that are adjacent in sequence to each other. Based on this, the execution processor and pre-order processor of each computing subtask are obtained, which provides a basis for the subsequent generation of scheduling code.

[0276] Fig.16 The embodiment of the present application provides a flow chart for generating corresponding scheduling codes based on the pre-order processor and execution processor of each computing subtask to obtain a preset number of scheduling codes, which may include steps 1601 to 1603. These steps are described in detail below.

[0277] 1601. Determine a coding position corresponding to each processor of at least two processors;

[0278] The encoding position of each processor in the SOC is preset.

[0279] The order of the processors in the SOC is determined in advance, and the encoding position is determined according to the order.

[0280] As an example, there are three processors in the SOC, namely GPU, CPU and NPU, in the order of GPU, CPU, NPU.

[0281] 1602. Set a first target code position in a first part of codes in the scheduling code corresponding to each computing subtask to a first value, and set a non-first target code position in the first part of codes to a second value, wherein the first target code position is a code position corresponding to a pre-order processor of the computing subtask;

[0282] Among them, the first part of the code in the scheduling code corresponding to the computing subtask is the part of the code corresponding to the preamble processor. According to the code positions corresponding to each processor, the value of the code position corresponding to the preamble processor of the computing subtask is set to the first value, and the values ​​of other code positions are set to the second value.

[0283] In a possible implementation, a code of 1 represents use of the processor, and a code of 0 represents not use of the processor.

[0284] As an example, the coding positions corresponding to the three processors GPU, CPU and NPU in the SOC are arranged in sequence. If the preceding processor of the computing task is the GPU, the first part of the code in its scheduling code is 100; if the preceding processor of the computing subtask is the GPU and the NPU, the first part of the code in its scheduling code is 101.

[0285] 1603. Set the second target coding position in the second part of the scheduling code corresponding to the computing subtask to the first value, and set the non-second target coding position in the second part of the code to the second value, where the second target coding position is the coding position corresponding to the execution processor of the computing subtask.

[0286] Among them, the second part of the code in the scheduling code corresponding to the computing subtask is the part of the code corresponding to the execution processor. According to the code positions corresponding to each processor, the value of the code position corresponding to the execution processor of the computing subtask is set to the first value, and the values ​​of other code positions are set to the second value.

[0287] As an example, the coding positions corresponding to the three processors GPU, CPU and NPU in the SOC are arranged in sequence. If the execution processor of the computing task is the CPU, the second part of the code in its scheduling code is 010; if the execution processor of the computing subtask is the GPU and NPU, the second part of the code in its scheduling code is 101.

[0288] The scheduling code is composed of a first part of code and a second part of code, and the first part of code is before the second part of code.

[0289] For example, the format of the scheduling code is xxxxyyyy, wherein the number of x and y is the same, the number of x and y is determined by the number of processors, xxxx is the first part of the code, and yyyy is the second part of the code.

[0290] refer to Fig.15 The scheduling codes generated by the calculation subtasks split in are 11100100 and 01011000.

[0291] In this embodiment, after determining the pre-order processor and the execution processor of each computing subtask, the coding position corresponding to each processor in the multiple processors is determined; the first target coding position in the first part of the coding in the scheduling code corresponding to each computing subtask is set to a first value, and the non-first target coding position in the first part of the coding is set to a second value, and the first target coding position is the coding position corresponding to the pre-order processor of the computing subtask; the second target coding position in the second part of the coding in the scheduling code corresponding to the computing subtask is set to a first value, and the non-second target coding position in the second part of the coding is set to a second value, and the second target coding position is the coding position corresponding to the execution processor of the computing subtask, and a unified format is used to generate scheduling codes for each computing subunit, providing a basis for subsequent execution of heterogeneous computing based on each processor.

[0292] Fig.17It is a schematic diagram of an application scenario of a heterogeneous computing method provided by the present application, and the application scenario includes 4 processors and a scheduling device 1701, and the processors are NPU, GPU, DSP and CPU. Among them, the NPU execution data type is INT4 and INT8, and the NPU contains operators 1-3; the GPU execution data type is INT8, FP16 and FP32, and the NPU contains operators 4-6; the DSP execution data type is INT8 and INT16, and the NPU contains operators 7-9; the CPU execution data type is INT8, FP16 and FP32, and the CPU contains operators 10-12. Among them, the scheduling device 1701 includes a logic unit, a coding unit, a DMA (Data Memory Access, direct memory access) engine (engine) and a clock. The logic unit includes a coding logic circuit and an output logic circuit, and the coding unit stores a scheduling coding queue FIFO. There is a transmission path for task completion signals between the coding logic circuit and each processor, and there is a transmission path for task start signals between the output logic circuit and each processor. After the NPU completes the calculation subtask, it transmits the task completion signal NPUstop to the encoding logic circuit; after the GPU completes the calculation subtask, it transmits the task completion signal GPUstop to the encoding logic circuit; after the DSP completes the calculation subtask, it transmits the task completion signal DSPstop to the encoding logic circuit; after the CPU completes the calculation subtask, it transmits the task completion signal CPUstop to the encoding logic circuit. The output logic circuit can send a task start signal NPUstart to the NPU; the output logic circuit can send a task start signal GPUstart to the GPU; the output logic circuit can send a task start signal DSPstart to the DSP; the output logic circuit can send a task start signal CPUstart to the CPU. The encoding logic circuit outputs a pop signal (POP) to the encoding unit based on the received task completion signal, and the encoding unit outputs the scheduling code in the scheduling code queue to the output logic circuit. The output logic circuit determines the execution processor based on the second part of the scheduling code, and sends a task start signal to the execution processor. For example, the scheduling code is xxxx0100, and it is determined that the execution processor is the GPU, and a task start signal is sent to the GPU.

[0293] Among them, each processor is connected to the memory through the memory bus. Fig.17 The memory in the system uses SRAM / DDR, and the execution results obtained by each processor when executing the calculation subtask are stored in the storage address allocated to the processor in the memory.

[0294] The memory also stores all the scheduling codes of the computing tasks. When the scheduling code queue cannot complete all the scheduling codes that carry the computing tasks, the scheduling code can be first transmitted to the DMA engine in the scheduling device through the memory bus. Based on the free position of the scheduling code queue, the DMA engine pushes the scheduling code into the scheduling code queue. Fig.17 Examples of medium scheduling codes include xxxx0100, xxxx1000, xxxx0100, xxxx1000.

[0295] Among them, the scheduling device 1701 is also provided with a clock, and a clock signal is generated by driving the clock to drive the scheduling device to be turned off or enabled. When the scheduling device is enabled, the heterogeneous computing processing method in the embodiment of the present application is executed.

[0296] A heterogeneous computing processing method provided by an embodiment of the present application is introduced above, and a device for executing the above heterogeneous computing processing method will be introduced below.

[0297] Fig.18 Schematic diagram of a heterogeneous computing device provided in an embodiment of the present application. Fig.18 As shown, the heterogeneous computing processing device 1800 includes:

[0298] A receiving module 1801 is used to receive a task completion signal sent by the first processor, where the task completion signal indicates that the first processor has completed the first computing subtask;

[0299] An acquisition module 1802, configured to acquire a target scheduling code based on the task completion signal;

[0300] A generating module 1803, used for generating a task start signal according to the task completion signal and the target scheduling code;

[0301] The trigger module 1804 is used to trigger the second processor to execute the second computing subtask according to the subtask file based on the task start signal, and the second computing subtask and the first computing subtask belong to the same computing task.

[0302] In a possible implementation, the generating module includes:

[0303] A parsing unit, used for parsing the target scheduling code to obtain a first part code and a second part code of the target scheduling code;

[0304] A first determining unit is used to determine, based on the first part of the code and the task completion signal, that each pre-order processor completes each corresponding first computing subtask, the pre-order processor is a processor for executing the first computing subtask, the first computing subtask is a pre-order subtask of the second computing subtask, and the pre-order processor of the second computing subtask includes at least one first processor;

[0305] A second determining unit, configured to determine an execution processor of a second computing subtask according to the second partial code;

[0306] The first generating unit is used to generate a task start signal according to the execution processor corresponding to the second part of the code, and the task start signal is used to trigger the execution processor to execute the second computing subtask.

[0307] In a possible implementation, the method further includes:

[0308] An execution result acquisition module is used to control the second processor to obtain the execution result of the first computing subtask according to the corresponding storage address of the subtask file in a preset storage area after triggering the second processor to execute the second computing subtask according to the subtask file based on the task start signal, so as to execute the second computing subtask according to the execution result of the first computing subtask, and a storage address is pre-allocated for each computing subtask in the preset storage area.

[0309] In a possible implementation, there are multiple first computing subtasks to be completed before the second computing subtask, and the obtaining module is specifically used to:

[0310] Determining, based on the task completion signal, that the first first computing subtask corresponding to the second computing subtask is completed, and obtaining a target scheduling code in a local preset scheduling code queue in response to the task completion signal;

[0311] Accordingly, the trigger module is specifically used for:

[0312] In response to the task completion signal, the completion status of each first computing subtask is determined; based on determining that each first computing subtask is completed, a task start signal is generated according to the target scheduling code.

[0313] In a possible implementation, the method further includes:

[0314] The reading module is used to read at least one scheduling code from the scheduling codes stored in the preset memory and put it into the scheduling code queue based on the idle state in the scheduling code queue. The computing subtasks corresponding to the scheduling codes stored in the preset memory constitute a computing task.

[0315] In a possible implementation, a module is obtained, specifically for:

[0316] Determine that the first computing subtask is completed based on the task completion signal, and determine the target scheduling code corresponding to the second computing subtask that is subsequent to the first computing subtask according to a preset linked list sequence;

[0317] Accordingly, the trigger module is specifically used for:

[0318] In response to the task completion signal, the completion status of each first computing subtask is determined; based on determining that each first computing subtask is completed, a task start signal is generated according to the target scheduling code.

[0319] In a possible implementation, it further includes: a pre-processing module, used to form a scheduling code and a sub-task file;

[0320] Wherein, the preprocessing module includes:

[0321] A computing task obtaining unit, used for obtaining computing tasks;

[0322] A splitting unit is used to split the computing task to obtain at least two computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask; the execution processor triggers the execution of the computing subtask based on the completion of the processing of the computing subtask by the corresponding pre-order processor, and the pre-order processor corresponding to each computing subtask is the execution processor corresponding to the previous computing subtask;

[0323] A second generating unit is used to generate a corresponding subtask file for each computing subtask, and the pre-order processor and the execution processor respectively execute the corresponding computing subtask according to the corresponding subtask file;

[0324] The third generating unit is used to generate corresponding scheduling codes according to the preceding processor and the executing processor of each computing subtask, so as to obtain the scheduling codes corresponding to each computing subtask.

[0325] In a possible implementation, the split unit is specifically used for:

[0326] Obtaining computing parameters of each of at least two processors;

[0327] According to the calculation parameters corresponding to each calculation step in the calculation task, the calculation task is divided into at least two calculation subtasks, and each calculation subtask includes at least one calculation step;

[0328] The processor matching the calculation parameters is used as the execution processor of the corresponding calculation subtask;

[0329] Based on the calculation order of each calculation subtask, a preceding processor of each calculation subtask is determined.

[0330] In a possible implementation, the third generating unit is specifically configured to:

[0331] determining a coding position corresponding to each of at least two processors;

[0332] The first target code position in the first part of the scheduling code corresponding to each computing subtask is set to a first value, and the non-first target code position in the first part of the code is set to a second value, wherein the first target code position is the code position corresponding to the pre-order processor of the computing subtask;

[0333] The second target coding position in the second part of the scheduling code corresponding to the computing subtask is set to a first value, and the non-second target coding position in the second part of the code is set to a second value, where the second target coding position is the coding position corresponding to the execution processor of the computing subtask.

[0334] It should be noted that for the functional explanation of each component structure in a heterogeneous computing processing device provided in an embodiment of the present application, please refer to the explanation in the aforementioned method embodiment, which will not be repeated in this embodiment.

[0335] In this embodiment, the computing task is first split into various computing subtasks, and a scheduling code is set for each computing subtask. The target scheduling code is obtained by generating a task completion signal when the first processor completes the preceding computing subtask. A task start signal is generated based on the target scheduling code and the task completion signal. The second processor executes the subsequent second computing subtask based on the subtask file. By setting up a scheduling device, the computing subtask scheduling between the processors is performed without occupying the CPU, reducing the software complexity of the CPU. After reducing the participation of the CPU, the overall power consumption of the SOC is reduced. Moreover, by implementing the inter-core synchronization logic of each processor through encoding, the parallel capability of each processor can be improved, and the computing efficiency of the SOC in executing computing tasks can be improved.

[0336] The present application also provides an electronic device in an embodiment. Fig.19 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.19 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0337] like Fig.19As shown, the electronic device may include a processing device 1901 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1902 or a program loaded from a storage device 1908 to a random access memory (RAM) 1903. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1903. The processing device 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0338] Typically, the following devices may be connected to the I / O interface 1905: an input device 1906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1908 including, for example, a memory card, a hard disk, etc.; and a communication device 1909. The communication device 1909 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.19 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0339] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any heterogeneous computing processing method provided in the embodiment of the present application.

[0340] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any heterogeneous computing processing method provided in an embodiment of the present application.

[0341] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0342] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0343] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0344] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A heterogeneous computing processing method, comprising: Receiving a task completion signal sent by the first processor, wherein the task completion signal indicates that the first processor has completed the first computing subtask; Obtaining a target scheduling code based on the task completion signal; Generate a task start signal according to the task completion signal and the target scheduling code; The second processor is triggered based on the task start signal to execute a second computing subtask according to the subtask file, and the second computing subtask and the first computing subtask belong to the same computing task.

2. The heterogeneous computing processing method according to claim 1, wherein the step of generating a task start signal according to the task completion signal and the target scheduling code comprises: Parsing the target scheduling code to obtain a first part code and a second part code of the target scheduling code; Determining, based on the first partial code and the task completion signal, that each pre-order processor completes each corresponding first computing subtask, the pre-order processor is a processor for executing a first computing subtask, the first computing subtask is a pre-order subtask of the second computing subtask, and the pre-order processor of the second computing subtask includes at least one first processor; Determining an execution processor of a second computing subtask according to the second part of the code; The execution processor corresponding to the second part of the code generates a task start signal, and the task start signal is used to trigger the execution processor to execute the second computing subtask.

3. The heterogeneous computing processing method according to claim 2, after triggering the second processor to execute the second computing subtask according to the subtask file based on the task start signal, further comprising: Control the second processor to obtain the execution result of the first computing subtask according to the storage address corresponding to the subtask file in the preset storage area, so as to execute the second computing subtask according to the execution result of the first computing subtask, and the preset storage area is pre-allocated with a storage address for each computing subtask.

4. The heterogeneous computing processing method according to claim 1, wherein there are multiple first computing subtasks to be completed before the second computing subtask, and the obtaining the target scheduling code based on the task completion signal comprises: Determining, based on the task completion signal, that the first first computing subtask corresponding to the second computing subtask is completed, and obtaining a target scheduling code in a local preset scheduling code queue in response to the task completion signal; Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes: In response to the task completion signal, determining the completion status of each first computing subtask; Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

5. The heterogeneous computing processing method according to claim 4, further comprising: Based on the idle state in the scheduling code queue, at least one scheduling code is read from the scheduling codes stored in the preset memory and put into the scheduling code queue, and the computing subtasks corresponding to the scheduling codes stored in the preset memory constitute a computing task.

6. The heterogeneous computing processing method according to claim 1, obtaining a target scheduling code based on the task completion signal, comprising: Determine that the first computing subtask is completed based on the task completion signal, and determine the target scheduling code corresponding to the second computing subtask that follows the first computing subtask according to a preset linked list sequence; Accordingly, generating a task start signal according to the task completion signal and the target scheduling code includes: In response to the task completion signal, determining the completion status of each first computing subtask; Based on determining that each of the first computing subtasks is completed, a task start signal is generated according to the target scheduling code.

7. According to the heterogeneous computing processing method of claim 1, the scheduling code and the subtask file are formed by the following method: Get a computational task; Splitting the computing task to obtain at least two computing subtasks and a pre-order processor and an execution processor corresponding to each computing subtask; the execution processor triggers the execution of the computing subtask based on the completion of the processing of the computing subtask by the corresponding pre-order processor, and the pre-order processor corresponding to each computing subtask is the execution processor corresponding to the previous computing subtask; Generate a corresponding subtask file for each computing subtask, and the pre-processor and the execution processor respectively execute the corresponding computing subtask according to the corresponding subtask file; A corresponding scheduling code is generated according to the pre-order processor and the execution processor of each computing subtask, and a scheduling code corresponding to each computing subtask is obtained.

8. The heterogeneous computing processing method according to claim 7, wherein the splitting of the computing task to obtain at least two computing subtasks and a pre-processor and an execution processor corresponding to each computing subtask comprises: Obtaining computing parameters of each of at least two processors; According to the calculation parameters corresponding to each calculation step in the calculation task, the calculation task is divided into at least two calculation subtasks, and each calculation subtask includes at least one calculation step; The processor matching the calculation parameters is used as the execution processor of the corresponding calculation subtask; Based on the calculation order of each calculation subtask, a preceding processor of each calculation subtask is determined.

9. The heterogeneous computing processing method according to claim 7, wherein the generating corresponding scheduling codes according to the pre-order processor and the execution processor of each computing subtask to obtain a preset number of scheduling codes comprises: determining a coding position corresponding to each of at least two processors; Setting a first target code position in a first part of codes in the scheduling code corresponding to each computing subtask to a first value, and setting a non-first target code position in the first part of codes to a second value, wherein the first target code position is a code position corresponding to a pre-order processor of the computing subtask; The second target coding position in the second part of the scheduling code corresponding to the computing subtask is set to a first value, and the non-second target coding position in the second part of the code is set to a second value, where the second target coding position is the coding position corresponding to the execution processor of the computing subtask.

10. A heterogeneous computing processing device, comprising: A receiving module, configured to receive a task completion signal sent by the first processor, wherein the task completion signal indicates that the first processor has completed the first computing subtask; An acquisition module, used for acquiring a target scheduling code based on the task completion signal; A generating module, used for generating a task starting signal according to the task completion signal and the target scheduling code; A trigger module is used to trigger the second processor to execute a second computing subtask according to the subtask file based on the task start signal, and the second computing subtask and the first computing subtask belong to the same computing task.

Images