Operator execution method, apparatus, electronic device and storage medium

Abstract

This invention provides an operator execution method, apparatus, electronic device, and storage medium, relating to the field of artificial intelligence technology. The method includes: performing operator fusion processing on multiple single operators in a neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fused operators, or M fused operators and N single operators; generating operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator; and executing the target operator in parallel based on the parallel configuration information. Through the above method, parallel execution of computations among operators is achieved, improving the parallelism of computations among fused operators, as well as the parallelism of computations between operators within each fused operator, thereby improving operator execution efficiency and operator development efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an operator execution method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence (AI), neural networks have been widely used in various fields. Running a neural network requires the support of a large number of operators.

[0003] In related technologies, neural networks are basically run in eager mode, where operators are usually executed serially, and synchronization between operators is guaranteed by hardware: that is, this is determined by running the kernel function multiple times.

[0004] However, the sequential execution of operators is time-consuming, resulting in low operator development efficiency. Therefore, improving the execution efficiency of operators is an urgent problem to be solved. Summary of the Invention

[0005] To address the problems existing in the prior art, embodiments of the present invention provide an operator execution method, apparatus, electronic device, and storage medium.

[0006] This invention provides an operator execution method, comprising:

[0007] Multiple single operators in a neural network model are fused to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1;

[0008] Based on the target operator, operator execution configuration information is generated; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator;

[0009] The target operator is executed based on the operator execution configuration information.

[0010] Optionally, the operator execution configuration information includes at least one of the following:

[0011] The first operator execution configuration information is used to indicate the operator parallel execution strategy among the M fusion operators, and the operator parallel execution strategy between the M fusion operators and the N single operators;

[0012] The second operator execution configuration information is used to indicate the operator parallel execution strategy among the sub-operators within each fusion operator.

[0013] Optionally, executing the target operator based on the operator execution configuration information includes:

[0014] When the target operator includes the M fusion operators and the N single operators, an operator execution instruction is generated based on the first operator execution configuration information and the second operator execution configuration information;

[0015] The target operator is executed based on the operator execution instructions.

[0016] Optionally, executing the target operator based on the operator execution configuration information includes:

[0017] When the target operator includes the M fusion operators, an operator execution instruction is generated based on the second operator execution configuration information;

[0018] The target operator is executed based on the operator execution instructions.

[0019] Optionally, executing the target operator based on the operator execution instruction includes:

[0020] For each fusion operator, in response to the operator execution instruction, and based on the second operator execution configuration information, the second sub-operator is executed when the first sub-operator sends a first message to the second sub-operator; the output of the first sub-operator is the input of the second sub-operator, and the first message is used to characterize that at least a portion of the data associated with the first sub-operator has been executed by the first sub-operator;

[0021] Based on the first operator execution configuration information, when the target sub-operator in each fusion operator sends a second message to the N single operators, the N single operators are executed; the second message is used to indicate that the target sub-operator has completed the execution of at least a portion of the data associated with the target sub-operator, and the output of the target sub-operator is the input of the N single operators.

[0022] Optionally, executing the target operator based on the operator execution instruction includes:

[0023] For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the fourth sub-operator is executed when the third sub-operator sends a third message to the fourth sub-operator;

[0024] The output of the third sub-operator is the input of the fourth sub-operator, and the third message is used to indicate that at least a portion of the data associated with the third sub-operator has been executed.

[0025] Optionally, before executing the target operator, the method further includes:

[0026] For each fusion operator, the data associated with each sub-operator in the fusion operator is segmented based on a preset data segmentation strategy.

[0027] Optionally, executing the target operator based on the operator execution instruction includes:

[0028] Map the operator execution instructions from the logical space to the physical space of the target operator in the hardware device;

[0029] The target operator is executed based on the operator execution instructions in the physical space.

[0030] Optionally, before performing operator fusion processing on multiple single operators in the neural network model, the method further includes:

[0031] Based on multiple single operators in the neural network model, a computation graph corresponding to the neural network model is generated, and the computation graph is used to represent the data dependencies between each single operator.

[0032] The present invention also provides an operator execution device, comprising:

[0033] The fusion module is used to perform operator fusion processing on multiple single operators in the neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1;

[0034] The first generation module is used to generate operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator;

[0035] The execution module is used to execute the target operator based on the operator execution configuration information.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the operator execution method as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the operator execution method as described above.

[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the operator execution method as described above.

[0039] The operator execution method, apparatus, electronic device, and storage medium provided by this invention perform operator fusion processing on multiple single operators in a neural network model to generate a target operator corresponding to the neural network model. The target operator includes M fusion operators, or M fusion operators and N single operators. Then, based on the target operator, operator execution configuration information is generated. Based on the parallel execution strategy indicated by the operator execution configuration information, M fusion operators, or M fusion operators and N single operators are executed, thereby realizing parallel execution of computations among operators, improving the parallelism of computations among fusion operators, as well as the parallelism of computations between operators within each fusion operator, and thus improving operator execution efficiency and operator development efficiency. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts illustrating the operator execution method provided by the present invention;

[0042] Figure 2 This is one of the schematic diagrams of the calculation graph provided by the present invention;

[0043] Figure 3 This is a second schematic diagram of the calculation graph provided by the present invention;

[0044] Figure 4 This is the third schematic diagram of the calculation diagram provided by the present invention;

[0045] Figure 5 This is a schematic diagram of the kernel corresponding to the operator graph provided by the present invention;

[0046] Figure 6 This is the second flowchart of the operator execution method provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the operator execution device provided by the present invention;

[0048] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined with Figures 1 to 6 The operator execution method provided by this invention will be described in detail. Figure 1 This is one of the flowcharts illustrating the operator execution method provided by this invention. See also... Figure 1 As shown, the method includes steps 101-103, wherein:

[0051] Step 101: Perform operator fusion processing on multiple single operators in the neural network model to generate the target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1.

[0052] First, it should be noted that this invention is applied to the computation graph compilation scenario in the software stack of artificial intelligence chips. The execution subject of this invention can be any electronic device capable of executing operators, such as any kind of smartphone, smartwatch, desktop computer, laptop, etc.

[0053] In this embodiment of the invention, the neural network model can be applied to fields such as image recognition, speech processing, and natural language processing. The neural network model can be, for example, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, etc. Running a neural network requires multiple operators as support.

[0054] Optionally, before performing operator fusion processing on multiple single operators in the neural network model, the following steps also need to be performed:

[0055] Based on multiple single operators in the neural network model, a computation graph corresponding to the neural network model is generated, and the computation graph is used to represent the data dependencies between each single operator.

[0056] In this embodiment of the invention, before performing operator fusion processing on multiple single operators in a neural network model, it is necessary to group the single operators from the neural network model according to their interfaces to generate a computation graph.

[0057] The computation graph corresponding to a neural network model is a directed acyclic graph used to describe the operations. It has two main elements: nodes and edges. Each node can correspond to a single operator, such as a vector, matrix, or tensor. Edges represent operations, such as addition, subtraction, multiplication, division, and convolution.

[0058] The computation graph reflects the data dependencies between individual operators. Figure 2 This is one of the schematic diagrams of the computational graph provided by the present invention. Figure 2 In the computational graph shown, the outputs of operators 1 and 2 are the inputs of operator 3, the output of operator 4 is the input of operator 5, and the outputs of operators 3 and 5 are the inputs of operator 6.

[0059] This invention proposes a systematic operator synchronization mechanism in computation graph mode, which configures different operator synchronization strategies (i.e. parallel execution strategies between operators) at different computation graph compilation stages, thereby improving the parallelism of computation between operators.

[0060] Step 102: Based on the target operator, generate operator execution configuration information; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator.

[0061] Optionally, the operator execution configuration information includes at least one of the following:

[0062] a) First operator execution configuration information, used to indicate the operator parallel execution strategy among the M fusion operators, and the operator parallel execution strategy between the M fusion operators and the N single operators.

[0063] b) Second operator execution configuration information, used to indicate the operator parallel execution strategy between sub-operators within each fusion operator.

[0064] In the above embodiments, by using the first operator execution configuration information and the second operator execution configuration information, the parallel execution strategy between each fusion operator, the parallel execution strategy between each fusion operator and each single operator, and the parallel execution strategy between each sub-operator within each fusion operator can be determined, thereby improving the parallelism of computation between operators.

[0065] Step 103: Execute the target operator based on the operator execution configuration information.

[0066] The operator execution method provided by this invention generates a target operator corresponding to the neural network model by performing operator fusion processing on multiple single operators in the neural network model. The target operator includes M fusion operators, or M fusion operators and N single operators. Then, based on the target operator, operator execution configuration information is generated. Based on the parallel execution strategy indicated by the operator execution configuration information, the M fusion operators, or M fusion operators and N single operators are executed, thereby realizing parallel execution of computations among operators, improving the parallelism of computations among fusion operators, as well as the parallelism of computations between operators within each fusion operator, and thus improving operator execution efficiency and operator development efficiency.

[0067] Optionally, before executing the target operator, the method further includes:

[0068] For each fusion operator, the data associated with each sub-operator in the fusion operator is segmented based on a preset data segmentation strategy.

[0069] In this embodiment of the invention, the data segmentation strategy is a tiling strategy. The tiling strategy is a technique that utilizes shared memory on the Graphics Processing Unit (GPU) to reduce access to global memory, thereby improving the execution efficiency of kernel functions. By using the tiling strategy, the data associated with each sub-operator in each fusion operator can be segmented, thereby reducing the amount of data input to each sub-operator and effectively improving the execution efficiency of the entire fusion operator.

[0070] Specifically, by using the tiling strategy to segment the data associated with each sub-operator in the fusion operator, multiple data blocks associated with each sub-operator are obtained.

[0071] During the execution of each sub-operator, the input data for each sub-operator is the segmented data block, thereby increasing the data processing volume of each sub-operator.

[0072] Optionally, executing the target operator based on the operator execution configuration information specifically includes at least one of the following methods:

[0073] Method 1, specifically including steps 1)-2):

[0074] Step 1) When the target operator includes the M fusion operators and the N single operators, generate operator execution instructions based on the first operator execution configuration information and the second operator execution configuration information.

[0075] Step 2) Execute the target operator based on the operator execution instruction.

[0076] Optionally, the execution of the target operator based on the operator execution instruction can be achieved through the following steps [1]-[2]:

[0077] Step [1]: For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the second sub-operator is executed when the first sub-operator sends a first message to the second sub-operator; the output of the first sub-operator is the input of the second sub-operator, and the first message is used to characterize at least a portion of the data associated with the first sub-operator after the first sub-operator has finished executing;

[0078] Step [2]: Based on the configuration information of the first operator execution, when the target sub-operator in each fusion operator sends a second message to the N single operators, the N single operators are executed; the second message is used to indicate that the target sub-operator has completed the execution of at least a portion of the data associated with the target sub-operator, and the output of the target sub-operator is the input of the N single operators.

[0079] Figure 3 This is the second schematic diagram of the calculation graph provided by the present invention. Figure 3 The computational graph shown includes fusion operator a, fusion operator b, and single operator c. Fusion operator a includes sub-operator 1, sub-operator 2, and sub-operator 3; fusion operator b includes sub-operator 4 and sub-operator 5.

[0080] a) For suboperator 1 and suboperator 3: suboperator 1 is the first suboperator, and suboperator 3 is the second suboperator.

[0081] Based on the second operator configuration information, sub-operator 1 first executes at least one data block associated with it (each data block is obtained by segmenting the data associated with sub-operator 1 using a tiling strategy). After sub-operator 1 has executed at least a portion of the data, it sends a first message to sub-operator 3. Upon receiving the first message, sub-operator 3 begins executing at least one data block associated with it.

[0082] In the above implementation, sub-operator 1 and sub-operator 3 can be executed in parallel.

[0083] b) For suboperator 2 and suboperator 3: suboperator 2 is the first suboperator, and suboperator 3 is the second suboperator.

[0084] Based on the configuration information of the second operator, sub-operator 2 first executes at least one data block associated with it. After sub-operator 2 has completed at least a portion of the data execution, it sends a first message to sub-operator 3. Upon receiving the first message, sub-operator 3 begins executing at least one data block associated with it.

[0085] In the above implementation, sub-operator 2 and sub-operator 3 can be executed in parallel.

[0086] c) For suboperator 4 and suboperator 5: suboperator 4 is the first suboperator, and suboperator 5 is the second suboperator.

[0087] Based on the second operator configuration information, sub-operator 4 first executes at least one data block associated with it. After sub-operator 4 has completed at least a portion of the data execution, it sends a first message to sub-operator 5. Upon receiving the first message, sub-operator 5 begins executing at least one data block associated with it.

[0088] In the above implementation, suboperator 4 and suboperator 5 can be executed in parallel.

[0089] d) For suboperators 3, 5 and single operator c: suboperators 3 and 5 are target suboperators.

[0090] Based on the configuration information of the first operator, sub-operators 3 and 5 first execute at least one data block associated with them. After sub-operators 3 and 5 have completed at least a portion of the data execution, they send a second message to single operator c. Upon receiving the second message from sub-operators 3 and 5, single operator c begins executing at least one data block associated with it.

[0091] In the above implementation, sub-operator 3, sub-operator 5, and single operator c can be executed in parallel.

[0092] It should be noted that in the above implementation, different operators belong to different computing units. Therefore, the parallel execution between different operators can be understood as the parallel execution between different computing units, or the synchronization between different computing units.

[0093] For example: synchronization between Tcore and the vector engine; synchronization between Vector and the vector engine.

[0094] Method 2, specifically including steps 1)-2):

[0095] Step 1) If the target operator includes the M fusion operators, generate operator execution instructions based on the second operator execution configuration information;

[0096] Step 2) Execute the target operator based on the operator execution instruction.

[0097] Optionally, the execution of the target operator based on the operator execution instruction can be achieved through the following steps:

[0098] For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the fourth sub-operator is executed when the third sub-operator sends a third message to the fourth sub-operator;

[0099] The output of the third sub-operator is the input of the fourth sub-operator, and the third message is used to indicate that at least a portion of the data associated with the third sub-operator has been executed.

[0100] Figure 4 This is the third schematic diagram of the calculation graph provided by the present invention. Figure 4 The computational graph shown includes fusion operator a and fusion operator b. Fusion operator a contains sub-operator 1, sub-operator 2, and sub-operator 3; fusion operator b contains sub-operator 4 and sub-operator 5.

[0101] a) For suboperator 1 and suboperator 3: suboperator 1 is the third suboperator, and suboperator 3 is the fourth suboperator.

[0102] Based on the second operator configuration information, sub-operator 1 first executes at least one data block associated with it (each data block is obtained by segmenting the data associated with sub-operator 1 using a tiling strategy). After sub-operator 1 has executed at least a portion of the data, it sends a third message to sub-operator 3. Upon receiving the third message, sub-operator 3 begins executing at least one data block associated with it.

[0103] In the above implementation, sub-operator 1 and sub-operator 3 can be executed in parallel.

[0104] b) For suboperator 2 and suboperator 3: suboperator 2 is the third suboperator, and suboperator 3 is the fourth suboperator.

[0105] Based on the configuration information of the second operator, sub-operator 2 first executes at least one data block associated with it. After sub-operator 2 has completed at least part of the data execution, it sends a third message to sub-operator 3. Upon receiving the third message, sub-operator 3 begins executing at least one data block associated with it.

[0106] In the above implementation, sub-operator 2 and sub-operator 3 can be executed in parallel.

[0107] c) For suboperator 4 and suboperator 5: suboperator 4 is the third suboperator, and suboperator 5 is the fourth suboperator.

[0108] Based on the second operator configuration information, sub-operator 4 first executes at least one data block associated with it. After sub-operator 4 has completed at least a portion of the data execution, it sends a third message to sub-operator 5. Upon receiving the third message, sub-operator 5 begins executing at least one data block associated with it.

[0109] In the above implementation, suboperator 4 and suboperator 5 can be executed in parallel.

[0110] It should be noted that since there is no data dependency between suboperator 3 and suboperator 5, after suboperator 3 and suboperator 5 receive the third message, they can each execute at least one data block associated with themselves.

[0111] Optionally, in the above embodiments, the execution of the target operator based on the operator execution instruction can be achieved through the following steps:

[0112] Step 1) Map the operator execution instructions from the logical space to the physical space of the target operator in the hardware device;

[0113] Step 2) Execute the target operator based on the operator execution instructions in the physical space.

[0114] In this embodiment of the invention, after generating the operator execution instructions, the kernel corresponding to the operator graph can be generated, specifically as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of the kernel corresponding to the operator graph provided by this invention.

[0115] After mapping the operator execution instructions from the logical space to the physical space of the target operator in the hardware device, the kernel is run to ensure the parallel execution of the target operator.

[0116] Figure 6 This is the second flowchart illustrating the operator execution method provided by this invention. See also... Figure 6 As shown, the method includes steps 601-610, wherein:

[0117] Step 601: Based on multiple single operators in the neural network model, generate a computation graph corresponding to the neural network model, wherein the computation graph is used to represent the data dependencies between each single operator.

[0118] Step 602: Perform operator fusion processing on multiple single operators in the neural network model to generate the target operator corresponding to the neural network model; wherein, the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1.

[0119] Step 603: For each fusion operator, the data associated with each sub-operator in the fusion operator is segmented based on a preset data segmentation strategy.

[0120] Step 604: When the target operator includes M fusion operators and N single operators, generate operator execution instructions based on the first operator execution configuration information and the second operator execution configuration information.

[0121] Step 605: For each fusion operator, in response to the operator execution instruction, and based on the second operator execution configuration information, execute the second sub-operator when the first sub-operator sends a first message to the second sub-operator; wherein, the output of the first sub-operator is the input of the second sub-operator, the first message is used to characterize at least a portion of the data associated with the first sub-operator after the first sub-operator has finished executing; the second operator execution configuration information is used to indicate the operator parallel execution strategy between the sub-operators within each fusion operator.

[0122] Step 606: Based on the first operator execution configuration information, when the target sub-operator in each fusion operator sends a second message to N single operators, execute N single operators; wherein, the second message is used to characterize that at least part of the data associated with the target sub-operator has been executed by the target sub-operator, the output of the target sub-operator is the input of the N single operators, and the first operator execution configuration information is used to indicate the operator parallel execution strategy between M fusion operators, and the operator parallel execution strategy between M fusion operators and N single operators.

[0123] Step 607: If the target operator includes M fusion operators, generate operator execution instructions based on the second operator execution configuration information.

[0124] Step 608: For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the fourth sub-operator is executed when the third sub-operator sends a third message to the fourth sub-operator; wherein, the output of the third sub-operator is the input of the fourth sub-operator, and the third message is used to characterize at least a portion of the data associated with the third sub-operator after the third sub-operator has finished executing.

[0125] It should be noted that the execution order of steps 604-606 and steps 607-608 is not important.

[0126] Step 609: Map the operator execution instructions from the logical space to the physical space of the target operator in the hardware device.

[0127] Step 610: Execute the target operator based on the operator execution instructions in the physical space.

[0128] The operator execution device provided by the present invention is described below. The operator execution device described below can be referred to in correspondence with the operator execution method described above. Figure 7 This is a schematic diagram of the operator execution device provided by the present invention, as shown below. Figure 7As shown, the operator execution device 700 includes: a fusion module 701, a first generation module 702, and an execution module 703, wherein:

[0129] The fusion module 701 is used to perform operator fusion processing on multiple single operators in the neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1;

[0130] The first generation module 702 is used to generate operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator;

[0131] The execution module 703 is used to execute the target operator based on the operator execution configuration information.

[0132] The operator execution device provided by this invention generates a target operator corresponding to the neural network model by performing operator fusion processing on multiple single operators in the neural network model. The target operator includes M fusion operators, or M fusion operators and N single operators. Then, based on the target operator, operator execution configuration information is generated. Based on the parallel execution strategy indicated by the operator execution configuration information, the M fusion operators, or M fusion operators and N single operators are executed, thereby realizing parallel execution of calculations between operators, improving the parallelism of calculations between fusion operators, as well as the parallelism of calculations between operators within each fusion operator, and thus improving operator execution efficiency and operator development efficiency.

[0133] Optionally, the operator execution configuration information includes at least one of the following:

[0134] The first operator execution configuration information is used to indicate the operator parallel execution strategy among the M fusion operators, and the operator parallel execution strategy between the M fusion operators and the N single operators;

[0135] The second operator execution configuration information is used to indicate the operator parallel execution strategy among the sub-operators within each fusion operator.

[0136] Optionally, the execution module 703 is further configured to:

[0137] When the target operator includes the M fusion operators and the N single operators, an operator execution instruction is generated based on the first operator execution configuration information and the second operator execution configuration information;

[0138] The target operator is executed based on the operator execution instructions.

[0139] Optionally, the execution module 703 is further configured to:

[0140] When the target operator includes the M fusion operators, an operator execution instruction is generated based on the second operator execution configuration information;

[0141] The target operator is executed based on the operator execution instructions.

[0142] Optionally, the execution module 703 is further configured to:

[0143] For each fusion operator, in response to the operator execution instruction, and based on the second operator execution configuration information, the second sub-operator is executed when the first sub-operator sends a first message to the second sub-operator; the output of the first sub-operator is the input of the second sub-operator, and the first message is used to characterize that at least a portion of the data associated with the first sub-operator has been executed by the first sub-operator;

[0144] Based on the first operator execution configuration information, when the target sub-operator in each fusion operator sends a second message to the N single operators, the N single operators are executed; the second message is used to indicate that the target sub-operator has completed the execution of at least a portion of the data associated with the target sub-operator, and the output of the target sub-operator is the input of the N single operators.

[0145] Optionally, the execution module 703 is further configured to:

[0146] For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the fourth sub-operator is executed when the third sub-operator sends a third message to the fourth sub-operator;

[0147] The output of the third sub-operator is the input of the fourth sub-operator, and the third message is used to indicate that at least a portion of the data associated with the third sub-operator has been executed.

[0148] Optionally, the device further includes:

[0149] The segmentation module is used to segment the data associated with each sub-operator in the fusion operator based on a preset data segmentation strategy for each fusion operator.

[0150] Optionally, the execution module 703 is further configured to:

[0151] Map the operator execution instructions from the logical space to the physical space of the target operator in the hardware device;

[0152] The target operator is executed based on the operator execution instructions in the physical space.

[0153] Optionally, the device further includes:

[0154] The second generation module is used to generate a computation graph corresponding to the neural network model based on multiple single operators in the neural network model. The computation graph is used to represent the data dependencies between the single operators.

[0155] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an operator execution method, which includes: performing operator fusion processing on multiple single operators in a neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1; generating operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator; and executing the target operator based on the operator execution configuration information.

[0156] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the operator execution method provided by the above methods. The method includes: performing operator fusion processing on multiple single operators in a neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1; generating operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate the parallel execution strategy of the target operator; and executing the target operator based on the operator execution configuration information.

[0158] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an operator execution method provided by the above methods. The method includes: performing operator fusion processing on multiple single operators in a neural network model to generate a target operator corresponding to the neural network model; the target operator includes M fusion operators, or M fusion operators and N single operators; M and N are integers greater than or equal to 1; generating operator execution configuration information based on the target operator; the operator execution configuration information is used to indicate a parallel execution strategy for the target operator; and executing the target operator based on the operator execution configuration information.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operator execution method, characterized in that, include: Multiple single operators in a neural network model are fused to generate the target operator corresponding to the neural network model. The target operator includes M fusion operators and N single operators; M and N are integers greater than or equal to 1; Based on the target operator, operator execution configuration information is generated, wherein the operator execution configuration information includes the following: The first operator execution configuration information is used to indicate the parallel execution strategy between the M fusion operators and between the M fusion operators and the N single operators; The second operator execution configuration information is used to indicate the intra-operator parallel execution strategy between sub-operators within each fusion operator; Based on the operator execution configuration information, the target operator is executed in parallel, wherein the parallel execution includes: Based on the configuration information of the second operator execution, within each fusion operator, after the first sub-operator has finished executing at least a portion of the data associated with the first sub-operator, it sends a message to the second sub-operator to trigger the second sub-operator to start execution, and the output of the first sub-operator is the input of the second sub-operator.

2. The operator execution method according to claim 1, characterized in that, The parallel execution of the target operator based on the operator execution configuration information includes: When the target operator includes the M fusion operators and the N single operators, an operator execution instruction is generated based on the first operator execution configuration information and the second operator execution configuration information; The target operator is executed in parallel based on the operator execution instructions.

3. The operator execution method according to claim 1, characterized in that, The parallel execution of the target operator based on the operator execution configuration information includes: When the target operator includes the M fusion operators, an operator execution instruction is generated based on the second operator execution configuration information; The target operator is executed in parallel based on the operator execution instructions.

4. The operator execution method according to claim 2, characterized in that, The parallel execution of the target operator based on the operator execution instructions includes: For each fusion operator, in response to the operator execution instruction, and based on the second operator execution configuration information, the second sub-operator is executed when the first sub-operator sends a first message to the second sub-operator; the output of the first sub-operator is the input of the second sub-operator, and the first message is used to characterize that at least a portion of the data associated with the first sub-operator has been executed by the first sub-operator; Based on the first operator execution configuration information, when the target sub-operator in each fusion operator sends a second message to the N single operators, the N single operators are executed; the second message is used to indicate that the target sub-operator has completed the execution of at least a portion of the data associated with the target sub-operator, and the output of the target sub-operator is the input of the N single operators.

5. The operator execution method according to claim 3, characterized in that, The parallel execution of the target operator based on the operator execution instructions includes: For each fusion operator, in response to the operator execution instruction, based on the second operator execution configuration information, the fourth sub-operator is executed when the third sub-operator sends a third message to the fourth sub-operator; The output of the third sub-operator is the input of the fourth sub-operator, and the third message is used to indicate that at least a portion of the data associated with the third sub-operator has been executed.

6. The operator execution method according to any one of claims 1 to 5, characterized in that, Prior to the parallel execution of the target operator, the method further includes: For each fusion operator, the data associated with each sub-operator in the fusion operator is segmented based on a preset data segmentation strategy.

7. The operator execution method according to any one of claims 2 to 5, characterized in that, The parallel execution of the target operator based on the operator execution instructions includes: Map the operator execution instructions from the logical space to the physical space of the target operator in the hardware device; The target operator is executed in parallel based on the operator execution instructions in the physical space.

8. The operator execution method according to any one of claims 1 to 5, characterized in that, Before performing operator fusion processing on multiple single operators in the neural network model, the method further includes: Based on multiple single operators in the neural network model, a computation graph corresponding to the neural network model is generated, and the computation graph is used to represent the data dependencies between each single operator.

9. An operator execution device, characterized in that, include: The fusion module is used to perform operator fusion processing on multiple single operators in the neural network model to generate the target operator corresponding to the neural network model. The target operator includes M fusion operators and N single operators; M and N are integers greater than or equal to 1; The first generation module is used to generate operator execution configuration information based on the target operator, wherein the operator execution configuration information includes the following: The first operator execution configuration information is used to indicate the parallel execution strategy between the M fusion operators and between the M fusion operators and the N single operators; The second operator execution configuration information is used to indicate the intra-operator parallel execution strategy between sub-operators within each fusion operator; An execution module is configured to execute the target operator in parallel based on the operator execution configuration information, wherein the parallel execution includes: Based on the configuration information of the second operator execution, within each fusion operator, after the first sub-operator has finished executing at least a portion of the data associated with the first sub-operator, it sends a message to the second sub-operator to trigger the second sub-operator to start execution, and the output of the first sub-operator is the input of the second sub-operator.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the operator execution method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the operator execution method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the operator execution method as described in any one of claims 1 to 8.

Images