Computing task configuration method, computing task execution method, and electronic device

Abstract

A configuration method of a computing task, an execution method of the computing task, and an electronic device. The configuration method of the computing task includes adding a first instruction to an executable instruction sequence corresponding to a first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, and to cause the executable instruction sequence corresponding to the first subtask to perform a corresponding operation when the first subtask is activated; and adding a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set a second subtask to an activated state. In the configuration method and the execution method of the computing task, the first subtask and the second subtask are automatically triggered to execute, and the second subtask does not need to be actively activated in a control flow by a host-side processor when the second subtask is executed, thereby simplifying an application layer control flow, reducing a burden caused by control layer code, increasing performance, saving an overhead of scheduling of the host-side processor for a high-throughput and low-latency exclusive application scenario.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a method for configuring a computing task, an apparatus for configuring a computing task, a method for executing a computing task, an apparatus for executing a computing task, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] Parallel computing refers to the process of using multiple computing resources simultaneously to solve computational problems, and is an effective means of improving the computing speed and data processing capabilities of computer systems. The basic idea is to decompose the problem to be solved into several parts, each of which is computed in parallel by an independent processing unit.

[0003] GPGPU (General Purpose Graph Processing Unit) is designed for parallel processing and can be used for high-performance parallel processing. It can compute different parts of the same task simultaneously and solve large problems that cannot be solved by a single processing unit. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a method for configuring a computing task, wherein the computing task includes multiple subtasks, the multiple subtasks including a first subtask and a second subtask, the first subtask and the second subtask being executed sequentially, and the configuration method including: adding a first instruction to an executable instruction sequence corresponding to the first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation; adding a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set the second subtask to an active state.

[0005] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided. When the first instruction is executed, it accesses the first control block data corresponding to the first subtask, cyclically checks the activation flag bit in the first control block data, and determines that the first subtask is activated in response to detecting that the activation flag bit in the first control block data is in a valid state.

[0006] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided. When the second instruction is executed, it accesses the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtains the second control block data, and sets the activation flag bit in the second control block data to be in an active state.

[0007] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, wherein when the second instruction is executed, the computing result data of the first subtask is also passed to the second subtask for further processing.

[0008] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided. When the second instruction is executed, it accesses a data storage address in the second control block data, wherein the data storage address is used to store data to be processed required to execute the second subtask; and selects an idle data storage address from the data storage address and stores the computing result data of the first subtask into the idle data storage address.

[0009] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, wherein the plurality of subtasks further includes M third subtasks, the M third subtasks and the second subtask are executed in parallel after the first subtask is executed, and when the second instruction is executed, the activation flag bits in the control block data corresponding to N of the M third subtasks are set to be in a valid state, wherein M is a positive integer and N is an integer less than or equal to M.

[0010] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided. When the second instruction is executed, the activation flags of MN third subtasks (excluding the N third subtasks) are set to an invalid state; or a preset data is transmitted to the MN third subtasks, wherein the preset data is used to indicate that when a third subtask receives the preset data, the executable instruction sequence corresponding to the third subtask does not need to be executed.

[0011] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, which includes adding a first instruction to the executable instruction sequence corresponding to the first subtask, including adding the first instruction before the executable instruction sequence.

[0012] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, which includes adding a second instruction to the executable instruction sequence corresponding to the first subtask, including adding the second instruction after the executable instruction sequence.

[0013] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided. The computing task is a computing task for a neural network, the neural network includes multiple fusion layers, and the computing task is executed according to the execution sequence corresponding to the neural network. The execution sequence represents the data dependency relationship and execution order between the multiple fusion layers. The multiple fusion layers correspond one-to-one with the multiple sub-tasks, and the executable instruction sequence corresponding to each sub-task is obtained by compiling the source program of the corresponding fusion layer.

[0014] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, in which a first instruction is added to the executable instruction sequence corresponding to the first subtask, including: obtaining the source program of a first operator, wherein the position of the first operator in the execution sequence is adjacent to and before the first fusion layer corresponding to the first subtask; and compiling the source program of the first operator to obtain the first instruction located before the executable instruction sequence.

[0015] For example, in at least one embodiment of this disclosure, a method for configuring a computing task is provided, in which a second instruction is added to the executable instruction sequence corresponding to the first subtask, including: obtaining the source program of the second operator, wherein the position of the second operator in the execution sequence is adjacent to and after the first fusion layer corresponding to the first subtask; and compiling the source program of the second operator to obtain the second instruction located after the executable instruction sequence.

[0016] At least one embodiment of this disclosure provides a method for executing a computing task, wherein the computing task includes multiple subtasks, the multiple subtasks including a first subtask and a second subtask, the first subtask and the second subtask being executed sequentially, and the execution method includes: executing a first instruction to determine whether the first subtask is activated; in response to the first subtask being activated, executing an executable instruction sequence corresponding to the first subtask; and after the executable instruction sequence is executed, executing a second instruction to set the second subtask to an active state.

[0017] For example, at least one embodiment of this disclosure provides a method for executing a computing task, which further includes: providing control block data corresponding one-to-one to the first subtask and the second subtask, wherein the control block data is used at least to determine whether the corresponding subtask is in an active state, and the control block data is stored in the storage space of the processor running the computing task.

[0018] For example, in at least one embodiment of this disclosure, a method for executing a computing task is provided, wherein the first control block data corresponding to the first subtask includes an activation flag bit, and a first instruction is executed to determine whether the first subtask is activated, including: cyclically checking the activation flag bit in the first control block data, determining that the first subtask is activated in response to the activation flag bit being in a valid state, and determining that the first subtask is not activated in response to the activation flag bit being in an invalid state.

[0019] For example, in at least one embodiment of this disclosure, a method for executing a computing task is provided. The first control block data corresponding to the first subtask includes a data storage address. The data storage address is used to store data to be processed required to execute the first subtask. In response to the activation of the first subtask, an executable instruction sequence corresponding to the first subtask is executed, including: extracting the data to be processed required to execute the first subtask according to the data storage address in the first control block data; and executing the executable instruction sequence corresponding to the first subtask in conjunction with the data to be processed.

[0020] For example, in at least one embodiment of this disclosure, a method for executing a computing task is provided, wherein the first control block data corresponding to the first subtask includes the control block storage address of the second control block data corresponding to the second subtask, and setting the second subtask to be in an active state includes: determining the second control block data according to the control block storage address; and setting the activation flag bit in the second control block data to be in a valid state.

[0021] For example, in at least one embodiment of this disclosure, a method for executing a computational task is provided. Before setting the second subtask to an active state after the executable instruction sequence has been executed, the execution method further includes: transmitting the computation result data of the first subtask to the second subtask for further processing.

[0022] For example, in at least one embodiment of this disclosure, a method for executing a computational task is provided. The first control block data corresponding to the first subtask includes the control block storage address of the second control block data corresponding to the second subtask. The second control block data includes a data storage address for storing the data to be processed required to execute the second subtask. The computation result data of the first subtask is passed to the second subtask for further processing, including: determining the second control block data according to the control block storage address; extracting the data storage address from the second control block data; selecting an idle data storage address from the data storage address; and storing the computation result data of the first subtask into the idle data storage address.

[0023] For example, at least one embodiment of this disclosure provides a method for executing a computing task, which further includes: after setting the second subtask to an active state, setting the first subtask to an inactive state, and continuing to execute the first instruction to determine whether the first subtask is activated.

[0024] For example, in at least one embodiment of this disclosure, a method for executing a computing task is provided, wherein the plurality of subtasks further includes M third subtasks, the M third subtasks and the second subtask are executed in parallel after the first subtask is executed, and the second subtask is set to be in an active state, and further includes: setting the activation flag bits in the control block data corresponding to N of the M third subtasks to be in a valid state, wherein M is a positive integer and N is an integer less than or equal to M.

[0025] At least one embodiment of this disclosure provides an electronic device, including a host-side processor and a device-side processor. The host-side processor is configured to control and schedule the device-side processor to execute computing tasks. The computing tasks include multiple subtasks, including a first subtask and a second subtask. The first subtask and the second subtask are executed sequentially. The device-side processor is configured to perform the following operations: execute a first instruction to determine whether the first subtask is activated; in response to the first subtask being activated, execute an executable instruction sequence corresponding to the first subtask; and after the executable instruction sequence is executed, execute a second instruction to set the second subtask to an active state.

[0026] For example, in an electronic device provided in at least one embodiment of this disclosure, the host-side processor is configured to activate the first executing start subtask among the plurality of subtasks and provide the initial data of the target computing task to the start subtask. The host-side processor is further configured to obtain the result data of the computing task from a result address, wherein the result address is configured to store the computing result data of the last executing end subtask among the plurality of subtasks.

[0027] For example, in at least one embodiment of the present disclosure, an electronic device is provided in which the storage space of the device-side processor stores first control block data corresponding to the first subtask and second control block data corresponding to the second subtask.

[0028] At least one embodiment of this disclosure provides a configuration device for a computing task, wherein the computing task includes multiple subtasks, the multiple subtasks including a first subtask and a second subtask, the first subtask and the second subtask being executed sequentially, and the configuration device includes: a first adding unit, configured to add a first instruction to an executable instruction sequence corresponding to the first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation; and a second adding unit, configured to add a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set the second subtask to an active state.

[0029] At least one embodiment of this disclosure provides an execution apparatus for a computing task, wherein the computing task includes multiple subtasks, the multiple subtasks including a first subtask and a second subtask, the first subtask and the second subtask being executed sequentially, and the execution apparatus includes: a first execution unit configured to execute a first instruction to determine whether the first subtask is activated; a second execution unit configured to execute an executable instruction sequence corresponding to the first subtask in response to the first subtask being activated; and a third execution unit configured to execute a second instruction after the executable instruction sequence has been executed to set the second subtask to an active state.

[0030] At least one embodiment of this disclosure provides an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions, when run by the processor, implement a configuration method for a computing task according to any embodiment of this disclosure or an execution method for a computing task according to any embodiment of this disclosure.

[0031] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement a configuration method for a computing task according to any embodiment of this disclosure or an execution method for a computing task according to any embodiment of this disclosure. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0033] Figure 1 This is a schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU);

[0034] Figure 2 A schematic flowchart illustrating a method for configuring a computing task according to at least one embodiment of this disclosure;

[0035] Figure 3A A schematic diagram of a directed acyclic graph of a neural network is shown;

[0036] Figure 3B It shows Figure 3A The execution sequence corresponding to the neural network shown;

[0037] Figure 4 A schematic diagram illustrating the allocation of computing and storage resources in a device-side processor according to an embodiment of this disclosure;

[0038] Figure 5 A schematic flowchart illustrating a method for executing a computing task according to at least one embodiment of this disclosure;

[0039] Figures 6-8 An execution flowchart of a method for executing a computing task provided in at least one embodiment of this disclosure;

[0040] Figure 9 A schematic block diagram of an electronic device provided for at least one embodiment of this disclosure;

[0041] Figure 10 A schematic block diagram of a configuration apparatus for a computing task provided for at least one embodiment of the present disclosure;

[0042] Figure 11 A schematic block diagram of an execution device for a computing task provided for at least one embodiment of the present disclosure;

[0043] Figure 12 A schematic diagram of an electronic device provided for at least one embodiment of this disclosure;

[0044] Figure 13 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0046] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0047] Figure 1 This is a schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU).

[0048] In parallel computing, computational tasks are typically executed using multiple threads. For example... Figure 1 As shown, before these threads execute in the general-purpose graphics processor (or parallel computing processor), they are divided into multiple thread blocks in the thread block scheduling module. Then, the thread block distribution module distributes these thread blocks to various computing units (CUs) (e.g., streaming multiprocessors (SMs)). All threads within a thread block must be assigned to the same computing unit for execution. Simultaneously, thread blocks are further divided into minimum execution thread warps (or simply thread warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute within the same computing unit or in different computing units.

[0049] Within each computing unit, the thread bundle scheduling / distribution module schedules and allocates thread bundles so that multiple computing cores (e.g., stream processors (SPs)) of that unit can run the thread bundles. Each computing core includes an arithmetic logic unit (ALU), a floating-point unit, etc. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Instruction fetching, decoding, and issuing are all completed within the thread bundle scheduling / distribution module. Memory execution instructions are issued to shared caches (e.g., shared L1 caches) within the computing unit or further issued to a unified cache for read / write operations, etc.

[0050] GPUs have a relatively simple structure, with numerous computing units and extremely long pipelines, making them particularly suitable for processing large amounts of uniformly typed data. However, GPUs cannot work independently; they must be controlled and scheduled by a Central Processing Unit (CPU) to operate. For example, when the CPU needs to process large amounts of uniformly typed data, it can invoke the GPU for parallel computing.

[0051] When a GPU performs parallel computing, the central processing unit (CPU) can divide the data into many blocks, each of which is processed by a stream in the GPU. Alternatively, it can be understood as the CPU breaking down the computational task into multiple subtasks, each processed by a stream in the GPU. Each stream involves one or more of the following three steps: 1) transferring data belonging to that stream from CPU memory to GPU memory; 2) the GPU performing the computation and storing the results in GPU memory; 3) copying the computational results of that stream from GPU memory to CPU memory.

[0052] Artificial Neural Networks (ANNs), also simply called neural networks, are mathematical models that mimic the behavioral characteristics of animal neural networks to perform distributed parallel information processing. These networks rely on the complexity of the system, adjusting the connections between a large number of internal nodes to achieve information processing. Neural networks can include Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Deep Belief Networks (DBNs), and Convolutional Neural Networks (CNNs), among others. Regardless of the type, artificial neural networks share common characteristics such as large-scale parallel processing, distributed storage, flexible topology, high redundancy, and nonlinear operations. They possess capabilities in areas such as processing speed, associative ability, adaptability, fault tolerance, and self-organization. These characteristics and capabilities form the technological foundation for artificial neural networks to simulate intelligent activities and have found important applications in various technological fields. For example, artificial neural networks can be used in data compression, image processing, video coding, and signal processing.

[0053] Due to hardware resource limitations of computing platforms and the demands of high-performance computing, multiple operators in a neural network that meet certain conditions or rules are typically fused together to form a fused operator in order to improve the computational efficiency of the neural network. A fused operator, or a single operator that cannot be fused, can be represented as a fusion layer, or simply a "layer". The computation process of the neural network is performed layer-by-layer, using fused layers as units.

[0054] For example, neural network computations can be performed on GPUs to accelerate the training and inference processes. For instance, when a neural network is computed on a GPU, it executes according to a corresponding execution sequence, which represents the data dependencies and execution order between multiple fusion layers in the neural network.

[0055] For example, the execution of a neural network's inference or training task on a GPU may consist of multiple subtasks, each corresponding to a fusion layer. These subtasks are executed sequentially according to the execution order defined by the execution sequence. Each subtask may perform at least some of the operations such as convolution, pooling, and normalization. During execution, a subtask may require the computational results of previously executed subtasks.

[0056] Currently, for computational tasks such as inference and training in neural networks, the data required by each subtask needs to be obtained through the host processor (e.g., a central processing unit). The host processor then transmits the data required by the subtask to the device processor (e.g., a general-purpose graphics processing unit). The device processor parses these data packets and schedules the computation units to perform the computation. Therefore, in scenarios where multiple subtasks need to be executed in a preset order, such as when the computation of a subtask requires the computation results of previously executed subtasks, each subtask needs to be actively triggered once in the control flow (i.e., in the CPU) during execution. According to the above-mentioned processing process in the GPU, after each subtask completes its computation, it sends the computation result data to the CPU (or stores it in the GPU's memory). The CPU then activates the next subtask and simultaneously passes the computation result data to that subtask. This results in a high difficulty in driver implementation, a significant performance degradation when multiple GPUs are scheduled simultaneously, and a heavy scheduling burden on the host processor.

[0057] This disclosure provides at least one embodiment of a method for configuring a computing task, an apparatus for configuring a computing task, a method for executing a computing task, an apparatus for executing a computing task, an electronic device, and a non-transitory computer-readable storage medium. The method for configuring a computing task includes: adding a first instruction to an executable instruction sequence corresponding to a first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation; and adding a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set a second subtask to an active state.

[0058] Corresponding to the configuration method of the computing task, at least one embodiment of this disclosure also provides a method for executing a computing task, including: executing a first instruction to determine whether a first subtask is activated; in response to the activation of the first subtask, executing an executable instruction sequence corresponding to the first subtask; and after the executable instruction sequence is executed, executing a second instruction to set the second subtask to be in an active state.

[0059] In the configuration and execution methods of this computing task, the first and second subtasks can be automatically triggered and executed sequentially. After the first subtask is completed, the second subtask is automatically activated. The second subtask does not require the host-side processor to actively activate it in the control flow, which simplifies the application layer control process, reduces the burden of control layer code, and increases performance. For high-throughput, low-latency exclusive application scenarios, it saves the overhead of host-side processor scheduling.

[0060] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0061] Figure 2 This is a schematic flowchart illustrating a method for configuring a computing task, provided for at least one embodiment of the present disclosure.

[0062] like Figure 2 As shown, at least one embodiment of this disclosure provides a method for configuring computing tasks, including steps S10 to S20.

[0063] For example, a computational task may include multiple subtasks, which may include a first subtask and a second subtask, and the first subtask and the second subtask may be executed sequentially.

[0064] For example, the first subtask and the second subtask may also be two subtasks that execute the configuration method for the computing task provided in at least one embodiment of this disclosure among a plurality of subtasks.

[0065] For example, multiple subtasks are executed sequentially according to the execution order. The first and second subtasks can be any two adjacent subtasks among the multiple subtasks. That is, the second subtask is the subtask that is adjacent to the first subtask in the execution order and is executed after the first subtask. For example, according to the definition of execution order, the second subtask needs to start executing immediately after the first subtask is executed. At this time, there may also be other subtasks that execute the configuration method after the second subtask or before the first subtask. The specific process of these subtasks executing the configuration method can be referred to the description of steps S10-S20 below.

[0066] For example, the computational task can be a computational task for a neural network, such as an inference or training task of the neural network, which includes multiple fusion layers, and the computational task is executed according to the execution sequence corresponding to the neural network.

[0067] Figure 3A A schematic diagram of a directed acyclic graph of a neural network is shown. Figure 3B It shows Figure 3A The execution sequence corresponding to the neural network shown.

[0068] A neural network can be viewed as a directed acyclic graph (DAG) consisting of many computational nodes, with each node corresponding to an operator. Figure 3A A directed acyclic graph (DAG) consisting of 13 computational nodes is shown, corresponding to a neural network with 13 operators. The nodes are connected by lines representing data dependencies and data flow. For example, the output data of node 1 flows to node 2, the output data of node 2 flows to nodes 3 and 6, and so on. This establishes a data dependency between node 1 and node 2, and between node 2 and nodes 6 and 3. It is understood that... Figure 3A The network structures shown are merely illustrative, and the methods according to embodiments of this disclosure can be applied to various types of neural network structures.

[0069] In related technologies, for neural network computation, in order to execute the computation process of operators in the network more efficiently, operator fusion can be performed according to certain rules and patterns, that is, merging multiple operators into a single fusion operator. For example... Figure 3A As shown, operators 1 and 2 can be merged into a single merge operator A; similarly, operators 3, 4, and 5 can be merged into a single merge operator C; operators 6, 7, 8, and 9 can be merged into a single merge operator B, and so on. Furthermore, individual operators that are not merged with other operators are also considered merge operators, such as merge operators D and E. Merge operators are also called merge layers. Of course, Figure 3A The illustration shows a schematic fusion process, but it is understood that the embodiments of this disclosure do not limit the specific rules for operator fusion, and any feasible method in the related technology can be used.

[0070] Next, according to Figure 3A The fused computation graph shown can be obtained Figure 3B The execution sequence shown is based on fusion layers (fusion operators) and is executed sequentially according to the data dependencies between fusion layers.

[0071] For example, when the computational task is for a neural network, multiple fusion layers correspond one-to-one with multiple subtasks. For instance, one fusion layer can be viewed as one subtask. Figure 3B For example, multiple subtasks include Figure 3B The fusion layer A (subtask A) to fusion layer F (subtask F) shown contains a total of 6 subtasks. For example, multiple subtasks are executed sequentially according to the execution sequence, with each subtask obtaining the computation results data of the executed subtasks based on data dependencies. For instance, when subtask C is executed, it needs the computation results data of subtask A, and when subtask E is executed, it needs the computation results data of subtasks B and C.

[0072] For example, the first subtask could be Figure 3B Subtask A in the context of subtasks, the second subtask could be... Figure 3B Subtask C or subtask B in the process. Of course, the first subtask can also be... Figure 3B Subtask C in the middle, the second subtask can be Figure 3B The subtasks D or E in the present disclosure are not specifically limited in this respect.

[0073] The following uses the first and second subtasks among multiple subtasks as examples to specifically illustrate the execution process of the computing task configuration method provided in at least one embodiment of this disclosure.

[0074] In step S10, a first instruction is added to the executable instruction sequence corresponding to the first subtask.

[0075] For example, the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs the corresponding operation.

[0076] In step S20, a second instruction is added to the executable instruction sequence corresponding to the first subtask.

[0077] For example, the second instruction is configured to set the second subtask to be active.

[0078] For example, the source code of each subtask can serve as a kernel function to be executed on the GPU. Compiling the source code of the subtask yields the underlying compilation result of the corresponding kernel function code, which is an executable instruction sequence, such as a binary instruction. The executable instruction sequence can include multiple binary instructions.

[0079] For example, for computational tasks used in neural networks, the executable instruction sequence corresponding to the subtask is obtained by compiling the source program of the corresponding fusion layer.

[0080] It should be noted that the first instruction and the second instruction in this disclosure are not included in the executable instruction sequence corresponding to the subtask. However, when the processor processes each subtask, the actual instruction sequence executed consists of the first instruction, the executable instruction sequence corresponding to the subtask, and the second instruction.

[0081] For example, when the first subtask is activated, it means that the first subtask has data that needs to be processed. When the first instruction is executed, if it is determined that the first subtask is activated, the sequence of executable instructions corresponding to the first subtask will be executed sequentially, thereby completing the corresponding operation of the first subtask, such as data processing, such as convolution, normalization, pooling and other computational operations.

[0082] For example, the executable instruction sequence corresponding to the second subtask also includes a first instruction and a second instruction with similar processing logic. The first instruction in the second subtask is always being executed. When the second instruction of the first subtask is executed, the second subtask is set to an active state. Thus, when the first instruction of the second subtask is executed, it detects that the second subtask is activated, and the executable instruction sequence corresponding to the second subtask begins to be executed sequentially, thereby completing the corresponding operation of the second subtask.

[0083] Similarly, multiple subtasks can further include third subtasks, fourth subtasks, and so on. For example, a second instruction added to the executable instruction sequence corresponding to the second subtask is used to set the third subtask to be active. The whole process follows the same logic. A first instruction added to each subtask is used to detect whether the subtask is active, that is, whether there is data to be processed. A second instruction added to the subtask is used to set the next subtask to be active after the first subtask. This achieves automatic sequential activation of multiple subtasks. The activation of each subtask does not require the host-side processor to actively activate it in the control flow, simplifying the application layer control flow, reducing the burden of control layer code, and increasing performance. For high-throughput, low-latency exclusive application scenarios, it saves the scheduling overhead of the host-side processor.

[0084] For example, in some embodiments, step S10 may include adding a first instruction before the executable instruction sequence.

[0085] For example, step S20 may include adding a second instruction after the executable instruction sequence.

[0086] For example, the first instruction and the second instruction may include one binary instruction, or they may include multiple binary instructions. This disclosure does not impose a specific limit on the number of binary instructions in the first instruction and the second instruction.

[0087] For example, an executable instruction sequence may include N binary instructions, a first instruction may be added before the first instruction in the N binary instructions, and a second instruction may be added after the Nth instruction in the N binary instructions.

[0088] In this embodiment, for example, the compiler can be modified to automatically add a first instruction before the executable instruction sequence and a second instruction after the executable instruction sequence when compiling the source program corresponding to the subtask.

[0089] For example, in other embodiments, the computation task is a computation task for a neural network, and step S10 may include: obtaining the source program of the first operator, wherein the position of the first operator in the execution sequence is adjacent to and before the first fusion layer corresponding to the first subtask; compiling the source program of the first operator to obtain a first instruction located before the executable instruction sequence.

[0090] Step S20 may include: obtaining the source program of the second operator, wherein the position of the second operator in the execution sequence is adjacent to and after the first fusion layer corresponding to the first subtask; compiling the source program of the second operator to obtain a second instruction located after the executable instruction sequence.

[0091] In this embodiment, a first operator for executing a first instruction and a second operator for executing a second instruction can be set at the source program level. For example, the first operator can be a self-looping operator. For instance, for each fusion layer in the execution sequence, a first operator is added before it, the function of which is to determine whether the subtask corresponding to that fusion layer is activated; a second operator is added after each fusion layer, the function of which is to set the subtasks corresponding to one or more fusion layers executed after that fusion layer to be in an active state. For example, with... Figure 3B For example, for fusion layers A to F, a first operator is added before fusion layer A and a second operator is added after fusion layer A. Similarly, a corresponding first operator is added before fusion layer B and a corresponding second operator is added after fusion layer B. Thus, after compilation by the compiler, the executable instruction sequences corresponding to fusion layers A to F can be obtained, as well as the first instruction before the executable instruction sequence corresponding to each fusion layer and the second instruction after the executable instruction sequence corresponding to each fusion layer.

[0092] In this embodiment, no modifications to the compiler are required; the compilation process adds a first instruction and a second instruction to each subtask at the source program level.

[0093] To achieve the functions described above for the first and second instructions, a one-to-one corresponding control block data can be set for each subtask. For example, the first subtask corresponds to the first control block data, and the second subtask corresponds to the second control block data. Alternatively, storage space can be pre-allocated in the device-side processor (e.g., GPU) to store the control block data corresponding to each subtask.

[0094] The following uses the first control block data as an example to illustrate the specific content of the control block data.

[0095] For example, the first control block data may include an activation flag, which can be a Boolean variable. This disclosure does not impose specific restrictions on the form or format of the activation flag. When the activation flag is valid (e.g., 1), it indicates that the first subtask corresponding to the first control block data is activated, and the first subtask has data to process. When the activation flag is invalid (e.g., 0), it indicates that the first subtask corresponding to the first control block data is not activated, and the first subtask has no data to process.

[0096] For example, the first control block data may also include the control block storage address of the second control block data corresponding to the second subtask. The second control block data can be found through the control block storage address.

[0097] For example, the first control block data may also include a data storage address, which is used to store the data to be processed required to execute the first subtask.

[0098] For example, in one possible embodiment, the structure of the control block data is defined as follows;

[0099] Struct ControlBlock{

[0100] bool is_valid;

[0101] void*next_ctr_block;

[0102] Queue data_addresses;

[0103] };

[0104] Here, "is_valid" represents the activation flag, which is a Boolean variable; "next_ctr_block" represents the control block storage address of the second control block data; and "data_address" represents the data storage address used to store the data required for the execution of the first subtask, for example, the data storage address is in queue form.

[0105] It should be noted that in some embodiments, multiple subtasks may be executed in parallel after the first subtask, for example, using... Figure 3B For example, after subtask C is completed, subtasks D and E can be executed. Therefore, for subtask C, the control block storage address in its corresponding control block data can include the control block storage addresses of the control block data corresponding to subtasks D and E.

[0106] Figure 4 This is a schematic diagram illustrating the allocation of computing and storage resources in a device-side processor according to an embodiment of the present disclosure.

[0107] like Figure 4 As shown, in the computing resources, kernel1 represents the executable instruction sequence corresponding to the first subtask, kernel2 represents the executable instruction sequence corresponding to the second subtask, and kernel3 represents the executable instruction sequence corresponding to the third subtask.

[0108] Referring to step S10 as described above, a first instruction "Head1" is added before kernel1, a first instruction "Head2" is added before kernel2, and a first instruction "Head3" is added before kernel3.

[0109] Referring to step S20 as described above, a second instruction "End1" is added after kernel1, a first instruction "End 2" is added after kernel2, and a first instruction "End 3" is added after kernel3.

[0110] Therefore, for the first subtask executed in the device-side processor, the actual execution in the computing resources is the first instruction "Head1", the executable instruction sequence kernel1, and the second instruction "End1". For the second subtask, the actual execution in the computing resources is the first instruction "Head2", the executable instruction sequence kernel2, and the second instruction "End2". For the third subtask, the actual execution in the computing resources is the first instruction "Head3", the executable instruction sequence kernel3, and the second instruction "End3".

[0111] like Figure 4 As shown, boxes with the same shaded area indicate resources belonging to the same subtask. For example, the first control block data corresponds to the first subtask, such as... Figure 4 As shown in the storage resources, the first control block data includes data storage address 1, data storage address 2, and data storage address 3. The data storage addresses indicate the addresses where the actual data is stored in the storage resources. For example, the data to be processed required by the first subtask is stored in the data block marked data 1 in the storage resources. Similarly, the second control block data corresponds to the second subtask, such as... Figure 4 As shown in the storage resources, the second control block data includes data storage address 4, data storage address 5, and data storage address 6. The data to be processed required by the second subtask is stored in the data block marked data 2 in the storage resources. The third subtask is similar and will not be described in detail here.

[0112] For example, when the first instruction is executed, it accesses the first control block data Head1 corresponding to the first subtask and cyclically checks the activation flag bit in the first control block data. Figure 4 (not shown in the image), in response to detecting that the activation flag bit in the first control block data is in a valid state, it is determined that the first subtask is activated.

[0113] For example, after the first subtask is activated, such as Figure 4 As shown, the executable instruction sequence kernel1 corresponding to the first subtask is executed sequentially. After the executable instruction sequence kernel is completed, the second instruction End1 is executed.

[0114] For example, when the second instruction End1 is executed, it accesses the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtains the second control block data, and sets the activation flag bit in the second control block data to be in a valid state.

[0115] For example, the first control block data stores the control block storage location of the second control block data. Based on the control block storage location, the second control block data can be determined, and the activation flag in the second control block data can be set to a valid state, thereby activating the second subtask and causing the second subtask to start executing the executable instruction sequence kernel2 corresponding to the second subtask.

[0116] In some embodiments, the computation result data of the first subtask also needs to be transmitted to the second subtask. For example, when the second instruction End1 is executed, it also transmits the computation result data of the first subtask to the second subtask for further processing.

[0117] For example, when the second instruction End1 is executed, it accesses the data storage address in the second control block data. This data storage address is used to store the data to be processed required to execute the second subtask. For example, it accesses... Figure 4 The data storage addresses are 4, 5, and 6; and an idle data storage address is selected from the data storage addresses to store the calculation result data of the first subtask into the idle data storage address. For example, if data storage address 4 is an idle data storage address, then the calculation result data of the first subtask can be stored into data storage address 4.

[0118] Since the first control block data contains the control block storage address of the second control block data, the second control block data can be found and determined through this control block storage address. Thus, the data storage address in the second control block data can be obtained, and the calculation result data in the first subtask can be stored in the free data storage address in the data storage address of the second control block data.

[0119] For example, such as Figure 4 As shown, the executable instruction sequences corresponding to each subtask occupy different computing resources. For example, the first and third subtasks occupy the same computing resources, both less than the computing resources occupied by the second subtask. In the computing task configuration method provided in at least one embodiment of this disclosure, the entire computing task can be allocated and adjusted according to the computing power allocation, computing speed, data throughput, etc. of each subtask, making it easier to combine computing power.

[0120] For example, in some embodiments, the multiple subtasks further include M third subtasks, and the M third subtasks and the second subtask are executed in parallel after the first subtask is executed.

[0121] For example, such as Figure 3B As shown, after subtask C is executed, subtasks D and E are executed in parallel. Therefore, subtask D can be the second subtask, and subtask E can be the third subtask.

[0122] At this point, when the second instruction is executed, it sets the activation flags in the control block data corresponding to N of the M third subtasks to a valid state, thus activating some of the subsequent parallel execution subtasks. For the MN third subtasks that do not need activation (excluding the N third subtasks), their activation flags can be set to an invalid state; alternatively, preset data (e.g., all zeros) can be transmitted to these MN third subtasks. For example, it can be pre-defined that when a subtask receives this preset data, the corresponding executable instruction sequence does not need to be executed. In this case, the activation flags of the MN third subtasks can be set to either a valid or invalid state. Here, M is a positive integer, and N is an integer less than or equal to M; for example, N can be equal to 0.

[0123] For example, in some embodiments, the first subtask can process 256 images simultaneously, such as performing pooling operations on the 256 images. The second subtask and 15 third subtasks receive these 256 processed image data and process them in parallel, such as performing convolution operations on the 256 processed image data in parallel. For example, the second subtask and each third subtask can process 16 processed image data. In some scenarios, the first subtask may not reach its peak processing volume, for example, the first subtask may only perform pooling operations on 16 images. In this case, the second subtask can be activated only in the manner described above, without activating the other 15 third subtasks. The computing resources used by the other 15 third subtasks can be used to perform other calculations. Compared to the current method, which can only activate all parallel subtasks simultaneously, the configuration method provided in this embodiment can save computing resources, reduce resource consumption, and improve computing efficiency.

[0124] By configuring the computational tasks as described above, a first instruction can be added to the beginning of the executable instruction sequence corresponding to each subtask, and a second instruction can be added to the end. The processing logic of the first and second instructions added to the executable instruction sequence corresponding to each subtask is similar. Thus, when executing each subtask, each subtask can be activated sequentially according to the execution order, and the computation result data of the subtask can be passed to one or more subtasks to be executed later according to the data dependency relationship. This reduces the application layer control flow, reduces the burden of the control layer code, and increases performance.

[0125] At least one embodiment of this disclosure also provides a method for executing a computing task, for example, applicable to the device side. Figure 5 This is a schematic flowchart illustrating a method for executing a computing task, provided for at least one embodiment of the present disclosure.

[0126] The computation task comprises multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. For a detailed description of the computation task, the first subtask, and the second subtask, please refer to the description of the computation task in the aforementioned configuration method; it will not be repeated here.

[0127] like Figure 5 As shown, at least one embodiment of this disclosure provides a method for executing a computing task, including steps S30 to S50.

[0128] In step S30, the first instruction is executed to determine whether the first subtask is activated.

[0129] In step S40, in response to the activation of the first subtask, the executable instruction sequence corresponding to the first subtask is executed.

[0130] In step S50, after the executable instruction sequence has been executed, the second instruction is executed to set the second subtask to an active state.

[0131] It should be noted that the specific execution process is explained here using the first subtask as an example. Of course, the computation task may also include the third subtask, the fourth subtask, etc., and their specific execution process is similar to that of the first subtask, so it will not be repeated here.

[0132] For example, at least one embodiment of this disclosure provides a method for executing a computing task that further includes providing control block data corresponding to a first subtask and a second subtask. For example, the control block data is used at least to determine whether the corresponding subtask is in an active state. The control block data is stored in the storage space of the processor running the computing task, for example, in the storage space of a device-side processor (e.g., a GPU). Because the control block data is stored in the storage space of the processor running the computing task, each subtask does not need to frequently retrieve data from the host-side processor when executing the first and second instructions. The host-side processor only needs to be activated once to achieve automatic execution of the entire data flow, simplifying the control process and reducing resource consumption.

[0133] For example, the control block data corresponding to the first subtask may include an activation flag, the control block storage address of the second control block data corresponding to the second subtask, and a data storage address. This data storage address is used to store the data to be processed required to execute the first subtask. For details regarding the specific content of the control block data, please refer to the relevant description in the aforementioned configuration method for computational tasks; details that are repeated will not be elaborated upon here.

[0134] For example, step S30 may include: cyclically checking the activation flag in the first control block data; determining that the first subtask is activated in response to the activation flag being in a valid state; and determining that the first subtask is not activated in response to the activation flag being in an invalid state.

[0135] For example, when the activation flag in the first control block data is detected to be in a valid state, the executable instruction sequence corresponding to the first subtask is executed to perform the corresponding operation.

[0136] For example, step S40 may include: extracting the data to be processed required to execute the first subtask based on the data storage address in the first control block data; and executing the executable instruction sequence corresponding to the first subtask in combination with the data to be processed.

[0137] For example, if the first subtask is the initiating subtask, the data to be processed can come from the initial data for computation provided by the host-side processor. For example, if the first subtask has previously executed other subtasks, the data to be processed can be the computation results of the previously executed subtasks, determined according to the execution order and data dependencies.

[0138] For example, with Figure 3B For example, for subtask A, the data to be processed can come from the initial data for computation provided by the host-side processor, and for subtask E, the data to be processed can come from the computation results of subtasks C and B.

[0139] Figure 6 This is an execution flowchart of a method for performing a computing task provided in at least one embodiment of the present disclosure. For example, Figure 6 The execution flowchart shown includes steps S30 and S40.

[0140] like Figure 6 As shown, for the first subtask, the activation flag bit in the first control block data is checked repeatedly. When the activation flag bit is in an invalid state, the activation flag bit in the first control block data is checked repeatedly. When the activation flag bit is in a valid state, the data storage address in the first control block data is obtained. The data to be processed is extracted according to the data storage address, and the executable instruction sequence of the first subtask is executed in combination with the data to be processed.

[0141] For example, the data storage address is a data structure in the form of a queue, where each element in the queue is a data storage address and corresponds to a data block of a preset size. When retrieving the data storage address, the address is obtained by updating the index at the end of the available data block. The specific process can be found in the relevant queue technologies, which will not be detailed here.

[0142] For example, step S50 may include: determining the second control block data according to the control block storage address; and setting the activation flag bit in the second control block data to be in a valid state.

[0143] For example, if it is necessary to transmit calculation result data, after the executable instruction sequence has been executed and before the second subtask is set to an active state, the execution method provided in at least one embodiment of this disclosure further includes: transmitting the calculation result data of the first subtask to the second subtask for further processing.

[0144] For example, passing the calculation result data of the first subtask to the second subtask may include: determining the second control block data based on the control block storage address; extracting the data storage address from the second control block data; selecting an idle data storage address from the data storage addresses, and storing the calculation result data of the first subtask into the idle data storage address.

[0145] For example, after setting the second subtask to an active state, the execution method of the computation task further includes: setting the first subtask to an inactive state and continuing to execute the first instruction to determine whether the first subtask is activated.

[0146] For example, after setting the second subtask to be active, the activation flag in the first control block data corresponding to the first subtask can be set to be inactive.

[0147] For example, setting the first subtask to an inactive state can be part of the functionality of the second instruction, executed together with the second instruction. For instance, the second instruction can be executed to set the first subtask to an inactive state after setting the second subtask to an active state.

[0148] Figure 7 This is an execution flowchart of a method for performing a computing task provided in at least one embodiment of the present disclosure. For example, Figure 7 The execution flowchart shown includes step S50.

[0149] like Figure 7 As shown, when executing the second instruction, the control block storage address in the first control block data is first read to obtain the second control block data.

[0150] Next, it is determined whether there is a free data storage address in the data storage address of the second control block data. If there is a free data storage address (for example, a data storage address is empty), the calculation result data of the first subtask is written into the free data storage address. Otherwise, it continues to wait until there is a free data storage address in the second control block data.

[0151] Then, the activation flag in the second control block data is set to be in an active state.

[0152] Then, the activation flag in the first control block data is set to an invalid state.

[0153] Thus, the execution of the second instruction is completed. Afterwards, the first subtask continues to execute the first instruction, waiting for the activation flag in the first control block data to be set to a valid state again. Furthermore, the second subtask is activated and begins executing its corresponding executable instruction sequence. The specific process is similar to steps S40 and S50, and will not be repeated here.

[0154] Figure 8 A flowchart illustrating a method for executing a computational task provided in at least one embodiment of this disclosure. Figure 8 The overall execution process of the execution method is shown.

[0155] like Figure 8 As shown, taking the first subtask as an example, the first instruction is executed first, and the specific process is as described in step S30, which will not be repeated here.

[0156] After the first subtask is activated, the executable instruction sequence corresponding to the first subtask is executed. The specific process is as described in step S40, and will not be repeated here.

[0157] Next, the second instruction is executed, and the specific process is as described in step S50, which will not be repeated here.

[0158] Then, continue executing the first instruction, and cyclically check the activation flag in the first control block data to confirm whether the first subtask has been reactivated.

[0159] For example, taking the first subtask as an example, similar logic is executed in parallel for other subtasks. For example, each subtask cyclically checks the activation flag in the control block data corresponding to each subtask. When the activation flag is detected to be in a valid state, the executable instruction sequence corresponding to that subtask and the corresponding second instruction are executed. This will not be elaborated further here.

[0160] In the computing task scheduling method provided in at least one embodiment of this disclosure, only the host-side processor needs to activate the initial subtask, and there is no need to actively activate each subtask in the control flow thereafter. This simplifies the application layer control flow, reduces the burden of control layer code, increases performance, and saves the overhead of host-side scheduling.

[0161] At least one embodiment of this disclosure also provides an electronic device. Figure 9 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0162] like Figure 9 As shown, the electronic device 100 includes a host-side processor 101 and a device-side processor 102.

[0163] For example, the host-side processor 101 may be a central processing unit, and the device-side processor 102 may be a graphics processor, such as a general-purpose graphics processor.

[0164] For example, the host-side processor 101 is configured to control and schedule the device-side processor to execute computing tasks. For example, a computing task may include multiple subtasks, such as a first subtask, a second subtask, ..., an Nth subtask. The first and second subtasks are executed sequentially, and the multiple subtasks are also executed sequentially according to their execution order. Figure 9 This example only provides one instance of a computation task and does not show the specific execution order or data dependencies. For more information on computation tasks, please refer to the description of their configuration methods; it will not be repeated here.

[0165] For example, the device-side processor 102 is configured to perform the following operations: execute a first instruction to determine whether a first subtask is activated; in response to the first subtask being activated, execute a sequence of executable instructions corresponding to the first subtask; and after the sequence of executable instructions has been executed, execute a second instruction to set the second subtask to be in an active state.

[0166] For details regarding the specific operations performed by the device-side processor 102, please refer to the relevant descriptions of steps S30 to S50 in the aforementioned method for executing computational tasks, which will not be repeated here. Furthermore, the device-side processor 102 can achieve similar technical effects to the aforementioned method for executing computational tasks, which will also not be repeated here.

[0167] For multiple subtasks, the computational task configuration method provided in at least one embodiment of this disclosure adds similar processing logic, such as the aforementioned first and second instructions, to the start and end positions of each subtask. Subsequently, each subtask is executed according to the aforementioned computational task execution method.

[0168] The following is combined Figure 9 This section provides a detailed explanation of the overall execution flow of the computation task.

[0169] For example, the host-side processor is configured to activate the first execution subtask among multiple subtasks and provide initial data for the target computation task to the starting subtask. For example, the starting subtask could be... Figure 9 The first subtask in the process.

[0170] For example, in some examples, an execution unit dedicated to data copying can be added before the starting subtask. This execution unit can be in the form of a kernel function. Its job is to check if there is a free data storage address in the data storage address of the control block data corresponding to the starting subtask. If there is a free storage address, the initial data is copied from the host-side processor.

[0171] For example, in other examples, the host-side processor can read the status of the data storage address in the control block data corresponding to the starting subtask, and copy the initial data to the free data storage address if there is one available.

[0172] For example, for the first subtask executed among multiple subtasks, such as Figure 9The first subtask in the process can have its activation flag set by the host-side processor. That is, once the host-side processor sets the activation flag of the starting subtask to be valid, multiple subtasks can be executed sequentially according to their execution order. The specific execution process can be found in the description of the aforementioned computation task execution method. Therefore, during the execution of the computation task, it is not necessary to actively activate each subtask, simplifying the application layer control flow.

[0173] Simultaneously, the host-side processor can assign values ​​to the control block storage addresses of the control block data corresponding to the next subtask to be executed, based on the execution order of multiple subtasks, and write these values ​​to the control block storage addresses of the control block data corresponding to the next subtask activated by each subtask. Thus, the execution order can be flexibly adjusted according to different computational tasks. After adjusting the control block storage addresses in each control block data, automatic pipelined execution of multiple subtasks with different execution orders can be achieved.

[0174] For example, the host-side processor is also configured to retrieve the result data of the computation task from a result address. For instance, the result address is configured to store the computation result data of the last executed subtask among multiple subtasks. For example, the terminating subtask could be... Figure 9 The Nth subtask in the process.

[0175] For example, for the last subtask executed among multiple subtasks, the calculation result data of the last subtask is the final calculation result of the calculation task, and this calculation result can be stored in the result address.

[0176] For example, in some cases, the device-side processor can read the result address and then initiate a copy of the data block corresponding to the result address, sending the data in the data block to the host-side processor.

[0177] For example, in other examples, the device-side processor sends a message to the host-side processor in the form of an event indicating that the computation task has been completed. The host-side processor then schedules and reads the data from the result address to obtain the result data of the computation task.

[0178] For example, the device-side processor's storage space stores the first control block data corresponding to the first subtask and the second control block data corresponding to the second subtask. Thus, the control block data corresponding to each subtask can be obtained directly from the device-side processor's storage space without needing to be transmitted by the host-side processor when activating each subtask, thereby enabling the automatic pipelined execution of multiple subtasks in the device-side processor.

[0179] At least one embodiment of this disclosure also provides a configuration apparatus for computing tasks, for example, suitable for the device side. Figure 10This is a schematic block diagram of a configuration apparatus for a computing task provided in at least one embodiment of the present disclosure.

[0180] For example, a computation task may consist of multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. For more information on computation tasks, please refer to the description of computation task configuration methods; details will not be repeated here.

[0181] like Figure 10 As shown, the configuration device 200 for computing tasks may include: a first addition unit 201 and a second addition unit 202.

[0182] For example, these units can be implemented through hardware (e.g., circuit) modules, software modules, or any combination of both, as is the case in the following embodiments, and will not be repeated here. For example, these units can be implemented through a central processing unit (CPU), a data processor (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions.

[0183] For example, the first adding unit 201 is used to add a first instruction to the executable instruction sequence corresponding to the first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation.

[0184] For example, the second adding unit 202 is used to add a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set the second subtask to an active state.

[0185] For example, the first adding unit 201 and the second adding unit 202 may include code and programs stored in memory; the processor may execute the code and programs to implement some or all of the functions of the first adding unit 201 and the second adding unit 202 as described above. For example, the first adding unit 201 and the second adding unit 202 may be dedicated hardware devices used to implement some or all of the functions of the first adding unit 201 and the second adding unit 202 as described above. For example, the first adding unit 201 and the second adding unit 202 may be a circuit board or a combination of multiple circuit boards used to implement the functions described above. In the embodiments of this application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in memory.

[0186] It should be noted that the first addition unit 201 can be used to implement Figure 2In step S10 shown, the second addition unit 202 can be used to implement... Figure 2 The step S20 is shown. Therefore, for a detailed explanation of the functions that the first adding unit 201 and the second adding unit 202 can achieve, please refer to the relevant descriptions of steps S10 to S20 in the embodiments of the above-described computing task configuration method; repeated details will not be repeated here. Furthermore, the computing task configuration device 200 can achieve similar technical effects to the aforementioned computing task configuration method, and will not be described further here.

[0187] It should be noted that, in the embodiments of this disclosure, the configuration device 200 for the computing task may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices according to circuit principles, or from digital chips, or in other suitable ways.

[0188] At least one embodiment of this disclosure also provides an apparatus for executing a computing task. Figure 11 This is a schematic block diagram of an execution device for a computing task provided for at least one embodiment of the present disclosure.

[0189] For example, a computation task may consist of multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. For more information on computation tasks, please refer to the description of computation task configuration methods; details will not be repeated here.

[0190] like Figure 11 As shown, the execution device 300 for computing tasks may include: a first execution unit 301, a second execution unit 302, and a third execution unit 303.

[0191] For example, these units can be implemented through hardware (e.g., circuit) modules, software modules, or any combination of both, as is the case in the following embodiments, and will not be repeated here. For example, these units can be implemented through a central processing unit (CPU), a data processor (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions.

[0192] For example, the first execution unit 301 is configured to execute a first instruction to determine whether the first subtask is activated.

[0193] The second execution unit 302 is configured to execute the executable instruction sequence corresponding to the first subtask in response to the activation of the first subtask.

[0194] The third execution unit 303 is configured to execute a second instruction after the executable instruction sequence has been executed, so as to set the second subtask to be in an active state.

[0195] For example, the first execution unit 301, the second execution unit 302, and the third execution unit 303 may include code and programs stored in memory; the processor may execute the code and programs to implement some or all of the functions of the first execution unit 301, the second execution unit 302, and the third execution unit 303 as described above. For example, the first execution unit 301, the second execution unit 302, and the third execution unit 303 may be dedicated hardware devices used to implement some or all of the functions of the first execution unit 301, the second execution unit 302, and the third execution unit 303 as described above. For example, the first execution unit 301, the second execution unit 302, and the third execution unit 303 may be a circuit board or a combination of multiple circuit boards used to implement the functions described above. In the embodiments of this application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in memory.

[0196] It should be noted that the first execution unit 301 can be used to implement Figure 5 The second execution unit 302 can be used to implement step S30 as shown. Figure 5 The third execution unit 303 can be used to implement step S40 shown. Figure 5 The step S50 is shown. Therefore, for a detailed description of the functions that the first execution unit 301, the second execution unit 302, and the third execution unit 303 can be found in the descriptions of steps S30 to S50 in the embodiments of the above-described computation task execution method; repeated descriptions will not be repeated here. Furthermore, the computation task execution device 300 can achieve similar technical effects to the aforementioned computation task execution method, and will not be described further here.

[0197] It should be noted that, in the embodiments of this disclosure, the execution device 300 for the computing task may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices according to circuit principles, or from digital chips, or in other suitable ways.

[0198] At least one embodiment of this disclosure also provides an electronic device. Figure 12 This is a schematic diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0199] For example, such as Figure 12As shown, the electronic device includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404. The processor 401, communication interface 402, and memory 403 communicate with each other via the communication bus 404. The processor 401, communication interface 402, and memory 403 can also communicate with each other via a network connection. This disclosure does not limit the type and function of the network. It should be noted that... Figure 9 The components of the electronic device shown are merely exemplary and not limiting; the electronic device may have other components depending on the actual application requirements.

[0200] For example, memory 403 is used to non-transitoryly store computer-readable instructions. When processor 401 executes the computer-readable instructions, it implements the configuration method or execution method of the computing task according to any of the above embodiments. For specific implementations and explanations of each step of the configuration method of the computing task, please refer to the embodiments of the configuration method of the computing task described above; for specific implementations and explanations of each step of the execution method of the computing task, please refer to the embodiments of the execution method of the computing task described above, and will not be repeated here.

[0201] For example, other implementations of the configuration method or execution method of the computing task implemented by the processor 401 executing computer-readable instructions stored in the memory 403 are the same as those mentioned in the foregoing method embodiment section, and will not be repeated here.

[0202] For example, communication bus 404 can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not mean that there is only one bus or one type of bus.

[0203] For example, communication interface 402 is used to enable communication between electronic devices and other devices.

[0204] For example, processor 401 can control other components in an electronic device to perform desired functions. Processor 401 can be a central processing unit (CPU), network processor (NP), tensor processor (TPU), or graphics processing unit (GPU) with data processing and / or program execution capabilities; it can also be a digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be based on x86 or ARM architectures, etc.

[0205] For example, memory 403 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and processor 401 may execute the computer-readable instructions to implement various functions of the electronic device. Various application programs and various data may also be stored in the storage medium.

[0206] For example, a detailed description of the process by which an electronic device performs data processing can be found in the relevant descriptions in the embodiments of the computing task configuration method and the computing task execution method, and repeated descriptions will not be repeated here.

[0207] Figure 13 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. For example, such as Figure 13 As shown, the storage medium 500 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 501 can be stored non-transitory. For example, when the computer-readable instructions 501 are executed by a processor, one or more steps in the configuration method of the computing task described above can be performed.

[0208] For example, the storage medium 500 can be used in the aforementioned electronic device, such as the storage medium 500 may include the memory in the electronic device.

[0209] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0210] For example, the description of storage medium 500 can be found in the description of memory in the embodiments of the electronic device, and will not be repeated here.

[0211] Those skilled in the art will understand that the contents disclosed herein can be varied and modified in many ways. For example, the various devices or components described above can be implemented in hardware, or in software, firmware, or a combination of some or all of the three.

[0212] Furthermore, while this disclosure makes various references to certain elements of systems according to embodiments of this disclosure, any number of different elements may be used and operated on clients and / or servers. Elements are merely illustrative, and different aspects of the system and method may use different elements.

[0213] This disclosure uses flowcharts to illustrate the steps of a method according to embodiments of this disclosure. It should be understood that the preceding or following steps are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes.

[0214] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This disclosure is not limited to any particular combination of hardware and software.

[0215] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms such as those defined in a common dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0216] The foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it. While several exemplary embodiments of the present disclosure have been described, those skilled in the art will readily understand that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A method for configuring a computing task, wherein, The computational task includes multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. The configuration method includes: A first instruction is added to the executable instruction sequence corresponding to the first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation; Add a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set the second subtask to an active state; When the first instruction is executed, it accesses the first control block data corresponding to the first subtask, cyclically checks the activation flag bit in the first control block data, and determines that the first subtask is activated in response to detecting that the activation flag bit in the first control block data is in a valid state. When the second instruction is executed, it accesses the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtains the second control block data, and sets the activation flag bit in the second control block data to the valid state. The executable instruction sequence consists of multiple binary instructions corresponding to the kernel function code obtained by compiling the source program of the first subtask, and the first instruction and the second instruction include at least one binary instruction.

2. The configuration method according to claim 1, wherein, When the second instruction is executed, it also passes the calculation result data of the first subtask to the second subtask for further processing.

3. The configuration method according to claim 2, wherein, When the second instruction is executed: Access the data storage address in the second control block data, wherein the data storage address is used to store the data to be processed required to execute the second subtask; and Select an idle data storage address from the data storage addresses, and store the calculation result data of the first subtask into the idle data storage address.

4. The configuration method according to claim 1, wherein, The plurality of subtasks also includes M third subtasks, which are executed in parallel with the second subtask after the first subtask is executed. When the second instruction is executed, the activation flag bit in the control block data corresponding to N of the M third subtasks is set to the valid state, where M is a positive integer and N is an integer less than or equal to M.

5. The configuration method according to claim 4, wherein, When the second instruction is executed, the activation flag of MN third subtasks other than the N third subtasks is set to an invalid state. or The preset data is transmitted to the MN third subtasks, wherein the preset data is used to indicate that when the third subtask receives the preset data, the executable instruction sequence corresponding to the third subtask does not need to be executed.

6. The configuration method according to any one of claims 1-5, wherein, Add a first instruction to the executable instruction sequence corresponding to the first subtask, including: The first instruction is added before the executable instruction sequence.

7. The configuration method according to claim 6, wherein, Add a second instruction to the executable instruction sequence corresponding to the first subtask, including: The second instruction is added after the executable instruction sequence.

8. The configuration method according to any one of claims 1-5, wherein, The computational task is a computational task for a neural network. The neural network includes multiple fusion layers. The computational task is executed according to the execution sequence corresponding to the neural network, wherein the execution sequence represents the data dependencies and execution order among the multiple fusion layers. The multiple fusion layers correspond one-to-one with the multiple subtasks, and the executable instruction sequence corresponding to each subtask is obtained by compiling the source program of the corresponding fusion layer.

9. The configuration method according to claim 8, wherein, Add a first instruction to the executable instruction sequence corresponding to the first subtask, including: Obtain the source program of the first operator, wherein the position of the first operator in the execution sequence is adjacent to and precedes the first fusion layer corresponding to the first subtask; The source program of the first operator is compiled to obtain the first instruction located before the executable instruction sequence.

10. The configuration method according to claim 8, wherein, Add a second instruction to the executable instruction sequence corresponding to the first subtask, including: Obtain the source program of the second operator, wherein the position of the second operator in the execution sequence is adjacent to and after the first fusion layer corresponding to the first subtask; The source program of the second operator is compiled to obtain the second instruction located after the executable instruction sequence.

11. A method for executing a computational task, wherein, The computational task includes multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. The execution method includes: Execute the first instruction to determine whether the first subtask is activated; In response to the activation of the first subtask, the executable instruction sequence corresponding to the first subtask is executed; and After the executable instruction sequence is completed, the second instruction is executed to set the second subtask to an active state; The execution method further includes: Provide one-to-one corresponding control block data for the first subtask and the second subtask, wherein the control block data is at least used to determine whether the corresponding subtask is in an active state, and the control block data is stored in the storage space of the processor running the computing task; The first control block data corresponding to the first subtask includes an activation flag. Execute the first instruction to determine whether the first subtask is activated, including: The activation flag bit in the first control block data is checked repeatedly. In response to the activation flag being in a valid state, it is determined that the first subtask is activated. In response to the activation flag being in an invalid state, it is determined that the first subtask is not activated; Execute the second instruction to set the second subtask to an active state, including: Access the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtain the second control block data, and set the activation flag bit in the second control block data to the valid state; The executable instruction sequence consists of multiple binary instructions corresponding to the kernel function code obtained by compiling the source program of the first subtask, and the first instruction and the second instruction include at least one binary instruction.

12. The execution method according to claim 11, wherein, The first control block data corresponding to the first subtask includes a data storage address, which is used to store the data to be processed required to execute the first subtask. In response to the activation of the first subtask, the executable instruction sequence corresponding to the first subtask is executed, including: Based on the data storage address in the first control block data, extract the data to be processed required to execute the first subtask; Based on the data to be processed, execute the executable instruction sequence corresponding to the first subtask.

13. The execution method according to claim 11, before setting the second subtask to an active state after the executable instruction sequence has been executed, the execution method further includes: The calculation result data of the first subtask is passed to the second subtask for further processing.

14. The execution method according to claim 13, wherein, The first control block data corresponding to the first subtask includes the control block storage address of the second control block data corresponding to the second subtask. The second control block data includes the data storage address for storing the data to be processed required to execute the second subtask. The calculation result data of the first subtask is passed to the second subtask for further processing, including: The second control block data is determined based on the control block storage address; Extract the data storage address from the second control block data; Select an idle data storage address from the data storage addresses, and store the calculation result data of the first subtask into the idle data storage address.

15. The execution method according to claim 11, further comprising: After setting the second subtask to an active state, the first subtask is set to an inactive state, and the first instruction continues to be executed to determine whether the first subtask is activated.

16. The execution method according to claim 11, wherein, The plurality of subtasks also includes M third subtasks, which are executed in parallel with the second subtask after the first subtask is executed. Setting the second subtask to be active also includes: The activation flag bits in the control block data corresponding to N of the M third subtasks are set to the valid state, where M is a positive integer and N is an integer less than or equal to M.

17. An electronic device, comprising a host-side processor and a device-side processor, The host-side processor is configured to control and schedule the device-side processor to execute computing tasks. in, The computational task includes multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. The device-side processor is configured to perform the following operations: Execute the first instruction to determine whether the first subtask is activated; In response to the activation of the first subtask, the executable instruction sequence corresponding to the first subtask is executed; as well as After the executable instruction sequence is completed, the second instruction is executed to set the second subtask to an active state; The storage space of the device-side processor stores the first control block data corresponding to the first subtask and the second control block data corresponding to the second subtask; The first control block data includes an activation flag. The device-side processor executes a first instruction to determine whether the first subtask is activated, including performing the following operations: The activation flag bit in the first control block data is checked repeatedly. In response to the activation flag being in a valid state, it is determined that the first subtask is activated. In response to the activation flag being in an invalid state, it is determined that the first subtask is not activated; The device-side processor executes a second instruction to set the second subtask to an active state, including performing the following operations: Access the control block storage location of the second control block data in the first control block data to obtain the second control block data, and set the activation flag bit in the second control block data to the valid state; The executable instruction sequence consists of multiple binary instructions corresponding to the kernel function code obtained by compiling the source program of the first subtask, and the first instruction and the second instruction include at least one binary instruction.

18. The electronic device according to claim 17, wherein, The host-side processor is configured to activate the first execution subtask among the plurality of subtasks, and to provide the initial data for the target computation task to the starting subtask. The host-side processor is further configured to obtain the result data of the computation task from the result address, wherein the result address is configured to store the computation result data of the last executed subtask among the plurality of subtasks.

19. A configuration device for a computing task, wherein, The computational task includes multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. The configuration device includes: The first adding unit is used to add a first instruction to the executable instruction sequence corresponding to the first subtask, wherein the first instruction is configured to determine whether the first subtask is activated, so that when the first subtask is activated, the executable instruction sequence corresponding to the first subtask performs a corresponding operation. The second adding unit is used to add a second instruction to the executable instruction sequence corresponding to the first subtask, wherein the second instruction is configured to set the second subtask to an active state; When the first instruction is executed, it accesses the first control block data corresponding to the first subtask, cyclically checks the activation flag in the first control block data, and determines that the first subtask is activated in response to detecting that the activation flag in the first control block data is in a valid state. When the second instruction is executed, it accesses the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtains the second control block data, and sets the activation flag bit in the second control block data to the valid state. The executable instruction sequence consists of multiple binary instructions corresponding to the kernel function code obtained by compiling the source program of the first subtask, and the first instruction and the second instruction include at least one binary instruction.

20. An execution device for a computational task, wherein, The computational task includes multiple subtasks, including a first subtask and a second subtask, which are executed sequentially. The actuator includes: The first execution unit is configured to execute a first instruction to determine whether the first subtask is activated; The second execution unit is configured to execute the executable instruction sequence corresponding to the first subtask in response to the activation of the first subtask; and The third execution unit is configured to execute a second instruction after the executable instruction sequence has been executed, so as to set the second subtask to be in an active state; The actuator is further configured to: Provide one-to-one corresponding control block data for the first subtask and the second subtask, wherein the control block data is at least used to determine whether the corresponding subtask is in an active state, and the control block data is stored in the storage space of the processor running the computing task; The first control block data corresponding to the first subtask includes an activation flag, and the first execution unit is configured as follows: The activation flag bit in the first control block data is checked repeatedly. In response to the activation flag being in a valid state, it is determined that the first subtask is activated. In response to the activation flag being in an invalid state, it is determined that the first subtask is not activated; The third execution unit is configured as follows: Access the control block storage location of the second control block data corresponding to the second subtask in the first control block data corresponding to the first subtask, obtain the second control block data, and set the activation flag bit in the second control block data to the valid state; The executable instruction sequence consists of multiple binary instructions corresponding to the kernel function code obtained by compiling the source program of the first subtask, and the first instruction and the second instruction include at least one binary instruction.

21. An electronic device, comprising: Memory stores computer-executable instructions non-transiently; The processor is configured to run computer-executable instructions. The computer-executable instructions are executed by the processor to implement the configuration method of the computing task according to any one of claims 1-10 or the execution method of the computing task according to any one of claims 11-16.

22. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the configuration method of the computing task according to any one of claims 1-10 or the execution method of the computing task according to any one of claims 11-16.

Images